Kyle Chui

Resource Management in Lua

Recently while working on nvim-surround, I ran into a scenario where I needed to manage resources; in particular extmarks. The only thing you really need to know about them is that they represent coordinates in a text buffer and get moved around as the buffer gets updated, which can be helpful for capturing "semantic" locations. Here's some pseudocode for how I was using them:

local extmark_id = set_extmark(initial_pos)
edit_buffer()
local extmark_pos = get_extmark(extmark_id)
del_extmark(extmark_id)
-- Do something with the "updated" position
-- ...

However, it began to irk me that the "garbage collection" for the extmarks wasn't automatic. Moreover, forgetting to clean them up could result in hundreds, if not thousands of extmarks being re-calculated with every buffer update. It occurred to me that I wanted bind the lifetime of the extmark to the lifetime of its ID1, to avoid all the normal "footguns" like use after free or invalid dereference. To accomplish this, I created a wrapper function that would handle instantiating and destroying the extmark automatically:

local with_extmark = function(initial_pos, callback)
  local extmark_id = set_extmark(initial_pos)
  callback()
  local extmark_pos = get_extmark(extmark_id)
  del_extmark(extmark_id)
  return extmark_pos
end

Then, whenever I needed to use an extmark, I could simply call with_extmark, without worrying about cleaning up any references:

local position = with_extmark(initial_pos, function()
  edit_buffer()
end)
-- Do something with the "updated" position
-- There are no extmark references visible here!

It's worth mentioning that none of the content here has been specific to Neovim extmarks; we could have instead discussed any other resource, like memory, mutexes, sockets, and more! As long as the language supports creating closures, we can use wrapper functions to ensure that no resources get leaked in our programs, avoiding a whole host of issues.


Footnotes

  1. In languages with constructors/destructors like C++, this can be done by binding an object's lifetime to a resource. In this case, creation of the object would "claim" the extmarks, and it "releases" the extmarks once it goes out of scope. This is also known as RAII.