Weeknotes 2022/15 - Dexie, SQL.js, and the Limits of LocalStorage
I started a new Slipmat.io sub-project where I’m working with DJ music collections and playlists. This introduces some interesting problems when done with Web technologies as we’re mostly talking about displaying lists of thousands, tens of thousands, or even hundreds of thousands of rows of data in one view.
Rendering super long lists is not common nor generally a smart idea in the browser in general, but this spesific case is more like the Finder on macOS than a typical Web app. In addition of just rendering these lists I also bumped into the limits of localStorage when trying to cache things. I’ve never had to worry about
QuotaExeededError exceptions with localStorage but when you’re trying to cache tens of thousands of lines of JSON, you’ll get there pretty fast.
Another interesting piece of technology is Dexie, which is a wrapper for IndexedDB. Dexie is already a pretty mature library and plays well with modern reactive tools like Vue and React.
While localStorage quota is in the range of 2-10 megabytes depending on the browser, IndexedDB is much higher, typically at least 1 Gb for desktop browsers. (Maximum storage depends on the browser and available disk space.) For my purposes I can easily fit everything I need into IndexedDB. Now the problem narrows down to being able to render huge lists in a way that feels smooth and doesn’t eat too much memory.