Carefully exploring Rust as a Python developer

I've recently joined a new company where one of the most used programming languages is Rust.

After using Python for almost a decade now, mostly for data engineering, I thought I'd give this new (to me) programming language a try and see if the praise I was reading about it on different platforms holds.

Rust is very different than Python and I won't elaborate on the technicalities that make Rust unique, there are many videos, articles and book written about the intricacies and technicalities of the language. Those pieces of content are excellent in explaining what Rust is but as a beginner I'm more focused in hitting the ground running as fast as possible.

I'm only focused on using the language to get stuff done with minimum friction but also evaluating at which degree I can figure out stuff on my own in order to learn. In some way, I'm more interested in complete dishes rather than the indexing of the ingredients properties.

Either way, I'd like to cover the basics before exploring more advanced topics.

Setting up a development environment

This has been straightforward by following the examples on the Rust website. Run a single command in a terminal and you're good to go. Everything should be configured and installed correctly.

To verify that Rust has been installed correctly, create an empty project in an empty directory with:  

cargo new tutorial

cd tutorial

cargo run

Now you can open that folder in your favourite text editor, I recommend VSCode if you're starting out as a few extensions that could help a lot and guides on how to use those extensions are very accessible. I recommend rust-analyzer as the only extension for VSCode.

Outputting and debugging stuff

I want to understand how my program is running and the first interaction with it is seeing what it's doing or what it has done through outputs to the command line.

It is also pretty much possible to use a regular debugger. On M1 I can recommend Visual Studio Code with LLDB, it works pretty well and is ultimately better than constantly printing things out to stdout (to each their own though).

So far it all looks very similar to what Python has to offer, except that the commands are run using cargo run instead of calling a specific file like python3 somefile.py.

Handling errors

Let's face it, shit happens and it's important to be able to deal with things in a predictable manner. A big challenge in programming is that it's hard to account for all the possible errors that can happen in a program since we added those errors ourselves by simply writing the code.

“Everyone knows that debugging is twice as hard as writing a program in the first place. So if you're as clever as you can be when you write it, how will you ever debug it?” - Brian W. Kernighan

In Python, this is usually done with try/except methods and raising exceptions. We try some stuff and if it fails we either try to match it to an exception we accounted for (a planned one) or we don't match it and add it to a generic exception. There are of course different categories of errors. While Python will let you bundle a package with a call to a non existing function on an object (and error on run time), Rust won't let you compile the code. "Runtime is fun time" doesn't hold when writing Rust so that eliminates a whole category of semi predictable errors.

While I can't fully explain the approach, I will attempt to explain this in Python terms through an example.

What is defined above is creating a custom exception that will be raised in the do_something function and checking for it in the main function. It's almost similar to a try/except but with a bit more boilerplate code.

All of the boilerplate code above is necessary to get it to work but is too complicated to understand at this stage of my learning. I mainly got it to work by following this guide, this one and this one.

I might hear you say now "there must be a better way", especially if you're writing Python for a while now and there indeed is. We'll have to use packages.

For a detailed explanation of the above check this out.

Using external packages

The cool thing about our industry is that we can reuse a lot of stuff that others built. One of the packages that is pretty interesting if you're planning out the failure modes of your program is thiserror. Rust leverages cargo as package manager.

Packages in Rust are called crates. They are installed by editing the Cargo.toml file located in the directory we created for this tutorial. In this case, we'd add thiserror = "1.0" right after [dependencies] and we're in business.

We can rewrite all the above code like this:

Ok now we're talking. This feels more like writing regular code while exploring Rust. I enjoy this a lot more than building everything from first principles. It took me 4 to 5 years to get to the more interesting parts in Python, so, advanced stuff for Rust can also wait a bit for when it's necessary. For more in-depth explanation on error handling, check this out. There are a lot of ways for how to handle errors in Rust, I prefer the lazy one :)

Obviously I'm skipping a few things like "what is an enum?", "what does that pub stand for?", "why are those hashtags for?" and that's on purpose. Their utility can be deduced from running the code, their understanding can be deferred after seeing them in action.

So far we managed quite a few things and to be honest, it's working out great. Let's figure out testing now shall we?

Writing tests

Tests should be configured and running on two levels: unit tests and integration tests. There are different ways on how to achieve this. While it's possible (and recommended by official guides) to put the tests in the same file than the rust code, what I like to do is to always have a separate folder with the tests neatly organised. This reduces the visual overload and screen bloat when editing something or making changes. And to be honest, I'm in a different mood when I write code than when I write tests (one of which involves red wine and cheese).

One way of doing it is like this:

You can run the above tests using cargo test resulting in the following:

There is a lot more to unpack here, but without making it too complicated, this is a "Pareto optimal" way of doing it (80/20). It also kinda reminds me of pytest, which immediately increases the level of comfyness.

For more details:

• Unit tests http://xion.io/post/code/rust-unit-test-placement.html
• Integration tests https://doc.rust-lang.org/rust-by-example/testing/integration_testing.html

Read a file, run something and write to another file

Now that we covered the basics: printing stuff out, debugging, using external packages and testing, we can do something a bit more productive. The canonical way of doing this is by writing a simple program that will process a local file. This is what I'd call the standard kata, get a CSV file, compute some metrics and write the results out.

For this to work, add csv = "1.1" and serde = { version = "1", features = ["derive"] } to your Cargo.toml. I let you guess how the main() method should look like.

Of course there is a lot more we can do with this. If you have a more complicated CSV file, you can use the pandas pendant in Rust called pola.rs to have more fun with the data. I still have to explore that a bit more but it seems extremely powerful and performant.

To be honest, when compared to how CSV parsing would work in Python I'd say it's roughly the same, except the automatic deserialising which I have to admit is pretty nice.  

We can add a few tests, but I'm not showing them here as it won't fit in a concise screenshot and won't add anything.

PS: consider this code "newbie" code.

Making HTTP requests

Moving up the hierarchy of a developer's needs, the next thing would be to make basic HTTP requests and do something with the results. Most of these requests process JSON nowadays so that's what we'll try.

Add the following three lines to your Cargo.toml and you're good to go: reqwest = { version = "0.11", features = ["json"] } and tokio = { version = "1", features = ["full"] } and serde_json = "1".

Ok now we're talking, we can request data from API's. As a data engineer this makes me very happy. Combining the above two methods we can fetch data, analyse it with pola.rs and write the results to a CSV relatively easy, with guarantees of memory safety and a lot more goodies. Noticed how the for loop reads like in Python? That's nice.

Now I'm sure the ecosystem will grow to cover a lot more use cases and adding or working with existing crates is a breeze. I've hit 0 walls writing this so far and looking things up as I go.

Work with SQLite

SQLite might be an odd addition to this list but I'm using it extensively in many of the programs or tools that I develop. I really like SQL since it's portable, efficient and requires exactly 0 maintenance.

The issue with using Python combined with SQL is always this question of using an ORM or not using an ORM. Don't get me wrong, SQLAlchemy does a great job but sometimes I feel like it's a bottleneck to the small things I'm developing. Add to that the complicated deployment steps to smaller embedded devices when using Python, it feels like the setup is doing a disservice to SQLite.

Which is where Rust comes in the picture. There are many examples online for how to use Rust in combination with SQLite and I believe it's a perfect combination.

A simple example would look like this, don't forget to add rusqlite = { version = "0.28.0", features = ["bundled"] } to Cargo.toml:

This example is copied from the rusqlite crate itself. It's a bit more up to date and accessible than the Rust Cookbook, which also has good examples.

Of course this only scratches the surface of what is possible. But combining only the methods above and without trying to be too smart about syntax and details, I'd wager it's possible to implement already a lot of useful things. All the while learning and getting initiated to a new programming language.  

Conclusion

All things considered, Rust is a solid language with a lot of very good practices baked in. I respect the efforts put in to make this wonderful language and I'll explore it more. I'm certain I'm not appreciating it to it's fullest - yet, but I'll get there.

While this was meant as a guide, I'm hoping for more "light touch" beginner guides to pop up to get more people interested to give this a go. I believe a programming language is only as good as whom it welcomes.

The first interaction with a programming language should be what it can do for you, rather than an exhaustive glossary of what it is. You really don't need to know in detail what borrowing, inheritance or traits are to do the stuff listed here but somehow too many guides just focus a lot on that. Those are great, but there's a wider audience (me included) that is just focused on building things that work, and can be maintained, and not that interested in theory and advanced concepts that don't seem to bring much.

Going from 0 to 1 is usually a lot more difficult than going from 1 to 10 :)

I'm aware this might be a bit confusing let me know if you think a video would be better to get the point across, I'm looking for reasons to make videos anyway.