Rust - Smart Ids; object lifetime and mutability
This article was previously published on len-learns-rust.com. A full index of the articles from this len-learns-rust.com can be found here.
The simple id manager that I built last time is just that, simple. However, it’s enough to start exploring some more complex ideas in Rust.
With the current interface you can allocate an id from the id manager and
never give it back. In fact, it’s easier to do that than it is to use it
properly and always give the id back when you’re done with it. The usual way
of addressing this issue in C++ is by using the RAII1 pattern whereby a
separate object becomes responsible for the lifetime of the resource. In this
case the resource is the id we’re allocating.
First we’ll create a ‘smart id’, don’t worry too much about the naming here, we can fix that later, what’s important is that the ‘smart id’ can own an id and know where to return it when it’s done with it. Our first attempt at this might look something like this:
pub struct SmartId {
manager: &mut IdManager,
id: u8,
we_own_id: bool,
}
However the compiler thinks otherwise…
error[E0106]: missing lifetime specifier
--> src\smart_id.rs:4:14
|
4 | manager: &mut IdManager,
| ^ expected named lifetime parameter
|
help: consider introducing a named lifetime parameter
|
3 ~ pub struct SmartId<'a> {
4 ~ manager: &'a mut IdManager,
|
So off I pop to read up about lifetime parameters2 and end up with:
pub struct SmartId<'a> {
manager: &'a mut IdManager,
id: u8,
we_own_id: bool,
}
Which tells the compiler that the id manager object has to live for at least the same length of time as the smart ids that you obtain from it. This makes perfect sense. In C++ I might have had a reference counted id manager that the smart id holds a reference to, but I probably wouldn’t have done… This is nicer as we’re forced to do the right thing.
The implementation is pretty simple, at least to start with:
impl<'a> SmartId<'a> {
pub fn new(manager: &'a mut IdManager) -> Self {
if !manager.can_allocate()
{
panic!("No Ids available")
}
let id = manager.allocate();
SmartId { manager, id, we_own_id: true }
}
pub fn release(&mut self) -> u8 {
self.we_own_id = false;
self.id
}
pub fn value(&self) -> &u8 {
&self.id
}
}
I could probably look into implementing dref() to replace value() but I like
the explicit nature of value() and I don’t fully understand dref(). This doesn’t
give me the RAII1 that I need though as we don’t have a ‘destructor’ that runs
when the object goes away. For that we need:
impl<'a> Drop for SmartId<'a> {
fn drop(&mut self) {
if self.we_own_id
{
self.manager.free(self.id);
}
}
}
With these tests, I can comment the Drop code in and out of existence and see
that it works correctly…
#[test]
fn test_create_one_smart_id() {
let mut manager = IdManager::new();
assert_eq!(manager.dump(), "[0,255]");
{
let id1 = SmartId::new(&mut manager);
let expected_id: u8 = 0;
assert_eq!(id1.value(), &expected_id);
//assert_eq!(manager.dump(), "[1,255]");
}
assert_eq!(manager.dump(), "[0,255]");
}
#[test]
fn test_release() {
let mut manager = IdManager::new();
assert_eq!(manager.dump(), "[0,255]");
{
let mut id = SmartId::new(&mut manager);
//assert_eq!(manager.dump(), "[1,255]");
id.release();
assert_eq!(manager.dump(), "[1,255]");
}
assert_eq!(manager.dump(), "[1,255]");
}
Unfortunately, if I uncomment those dump() checks things start to go wrong…
error[E0502]: cannot borrow `manager` as immutable because it is also borrowed as mutable
--> src\smart_id.rs:118:24
|
116 | let mut id = SmartId::new(&mut manager);
| ------------ mutable borrow occurs here
117 |
118 | assert_eq!(manager.dump(), "[1,255]");
| ^^^^^^^^^^^^^^ immutable borrow occurs here
119 |
120 | id.release();
| ------------ mutable borrow later used here
and this is just the start of the problems… When I add a test to create multiple smart ids:
#[test]
fn test_create_multiple_smart_ids() {
let mut manager = IdManager::new();
assert_eq!(manager.dump(), "[0,255]");
{
let id1 = SmartId::new(&mut manager);
let expected_id1: u8 = 0;
assert_eq!(id1.value(), &expected_id1);
//assert_eq!(manager.dump(), "[1,255]");
{
let mut id2 = SmartId::new(&mut manager);
let expected_id2: u8 = 1;
assert_eq!(id2.value(), &expected_id2);
//assert_eq!(manager.dump(), "[2,255]");
id2.release();
{
let id3 = SmartId::new(&mut manager);
let expected_id: u8 = 2;
assert_eq!(id3.value(), &expected_id);
//assert_eq!(manager.dump(), "[3,255]");
}
}
//assert_eq!(manager.dump(), "[2,255]");
}
//assert_eq!(manager.dump(), "[0], [2,255]");
}
Even creating two smart ids has the compiler complaining…
error[E0499]: cannot borrow `manager` as mutable more than once at a time
--> src\smart_id.rs:82:44
|
73 | let id1 = SmartId::new(&mut manager);
| ------------ first mutable borrow occurs here
...
82 | let mut id2 = SmartId::new(&mut manager);
| ^^^^^^^^^^^^ second mutable borrow occurs here
...
104 | }
| - first borrow might be used here, when `id1` is dropped and runs the `Drop` code for type `smart_id::SmartId`
The compiler warning above helped to point me in the right direction. Once again Rust is seeing what could happen rather than what does happen. In C++ this would have been OK but would have caused problems with multithreaded code if two smart ids tried to access the id manager at the same time. With Rust the compiler sees that this is a bit fast and loose and prevents it. This makes Rust harder to “sketch” with but often code that I “sketch” in C++ somehow ends up in production…
Due to my lack of understanding about the Rust I’m jumping to a conclusion here that, whilst almost right, is very much a simplistic solution. I’m using the biggest hammer in the toolbox on this nail. What we have here is the Interior Mutability Pattern and there are several ways to solve it, and to work nicely with the borrow checker without having to use full-blown locking. We need locking for using the id manager with multiple threads, but we could solve this issue in a simpler way if I understood more Rust at this point.
What’s more, as I found out here, we actually need more changes to make this code work nicely with multiple threads…
The code to this point can be found here so that you can easily compare it to the fixed version…
As before, the compiler pointed us in the right direction by suggesting that the
ids may be dropped simultaneously. My first thought here was that locking might
help and it does. In fact, it helps massively with getting the compiler to let us
do what we want to do, because, once we add the locking, the compiler trusts that
the code will actually do what we’re trying to make it do and will do it safely.
The fix here is to make sure that the smart id only works in terms of a lockable
id manager, that is one that is wrapped in a std::sync::Mutex<>.
So, something a bit like this, perhaps:
pub struct SmartId<'a> {
manager: &'a Mutex<IdManager>,
id: u8,
we_own_id: bool,
}
Note that the id manager is no longer mutable. This may seem strange but the
std::sync::Mutex<>
gives us the ability to convert a non-mutable reference to the id manager into
a mutable reference. This, in essence, is the solution to all of the issues that
we were having with the first attempt at the code. The implementation is now:
impl<'a> SmartId<'a> {
pub fn new(manager: &'a Mutex<IdManager>) -> Self {
let mut locked = manager.lock().unwrap();
if !locked.can_allocate()
{
panic!("No Ids available")
}
let id = locked.allocate();
SmartId { manager, id, we_own_id: true }
}
pub fn release(&mut self) -> u8 {
let _locked = self.manager.lock().unwrap();
self.we_own_id = false;
self.id
}
pub fn value(&self) -> &u8 {
&self.id
}
}
impl<'a> Drop for SmartId<'a> {
fn drop(&mut self) {
let mut locked = self.manager.lock().unwrap();
if self.we_own_id
{
locked.free(self.id);
}
}
}
Note that when we need a mutable reference to the id manager we obtain it
as a std::sync::MutexGuard<>
that provides a deref() implementation that returns a mutable id manager.
This means that the tests become more like this:
#[test]
fn test_create_one_smart_id() {
let manager = Mutex::new(IdManager::new());
assert_eq!(manager.lock().unwrap().dump(), "[0,255]");
{
let id1 = SmartId::new(&manager);
let expected_id: u8 = 0;
assert_eq!(id1.value(), &expected_id);
assert_eq!(manager.lock().unwrap().dump(), "[1,255]");
}
assert_eq!(manager.lock().unwrap().dump(), "[0,255]");
}
and the compiler is happy as we don’t have any overlapping borrows. The code’s horrible though and we’ll deal with that next.
The code to this point can be found here so that you can easily compare it to the fixed version…
Now that we have ugly code that ‘works’ we can make it easier to use. Again, ignore the naming used here, we can fix that later. This is part of my heritage showing through. With C++ I sometimes write a version of the code that ‘does the work’ but that cannot be safely used by multiple threads and then a ’thread safe’ wrapper that adds appropriate locking. With Rust the compiler doesn’t like the idea of the programmer deciding when it’s ok to use the unsafe version of the code. I think that’s a good thing. I’m often not the best person to make these decisions, or, more accurately, I often make mistakes when code evolves and suddenly needs to be thread safe…
So, now we’ll add a ’thread safe id manager’ which encapsulates the id manager
that we have already written and the std::sync::Mutex<>
that the smart id needs to convince the compiler that it’s always going to be
used correctly.
pub struct ThreadSafeIdManager {
manager : Mutex<IdManager>
}
which exposes simple implementations of the id manager functionality that lock the manager before using it, so stuff like this:
impl ThreadSafeIdManager {
pub fn new() -> Self {
let manager = Mutex::new(IdManager::new());
ThreadSafeIdManager { manager }
}
pub fn dump(&self) -> String {
let locked = self.lock();
locked.dump()
}
I then duplicate most of the id manager’s tests, which I’m not happy about
The smart id test looks like this:
#[test]
fn test_create_one_smart_id() {
let manager = ThreadSafeIdManager::new();
{
let _id = manager.allocate_smart_id();
assert_eq!(manager.dump(), "[1,255]");
}
assert_eq!(manager.dump(), "[0,255]");
}
These changes leave us with a usage example that looks like this:
extern crate idmanager;
use idmanager::thread_safe_id_manager::ThreadSafeIdManager;
pub fn main() {
let manager = ThreadSafeIdManager::new();
assert_eq!(manager.dump(), "[0,255]");
{
let id = manager.allocate_smart_id();
let expected_id : u8 = 0;
assert_eq!(id.value(), &expected_id);
assert_eq!(manager.dump(), "[1,255]");
}
assert_eq!(manager.dump(), "[0,255]");
}
Which, apart from names, is reasonably clean. We’ll deal with the naming issues next time.
The code can be found here on GitHub each step on the journey will have one or more separate directories of code, so this article’s code is here, here and here as this allows for easy comparison of changes at each stage.
Of course, there may be a better way; leave comments if you’d like to help me learn.
-
Lifetime parameters, here, in Rust by example, and here, in the book. ↩︎
