Why I created the Zetsu API
Hi, I’d like to show you one of the coolest services I’ve ever created because of its usefulness. It’s called Zetsu API and is a simple API tool that you can use to store and retrieve files using the Cloudflare network. The files are stored directly in the database, and this API is designed to be a microservice. The Zetsu is licensed under the GPLv3 which means you can host it yourself in your own Cloudflare account. The only things you need is a Cloudflare D1 database and a Cloudflare Worker.
Why I created the Zetsu API?
When I was maintaining my blog many years ago, I couldn’t find a way to host my images without worrying about committing them in a Git repository. First, I don’t like doing that because images take up too much space, and second, it didn’t feel “clean enough” for me. I feel like git repositories should only contain source code and binary assets should be hosted somewhere else. So, what other options I had?
- Image Kit
- Cloudflare R2
- Database storage
Many people don’t consider storing files in the database because that’s not the default choice in almost every project, they prefer to use a file service like Cloudflare R2 or Amazon S3 and store the URL in a database. I don’t like this approach because you have no guarantee that the data is correct, you might have a URL for a file that doesn’t exist and vice-versa. Plus, depending on the service you choose, you have to worry about file names, folder structures, and I didn’t want to deal with any of that. File storage services like ImageKit and Cloudflare R2 are fine, I’m not saying you shouldn’t use them, but when it comes to writing articles, I don’t give a damn about file paths, file names and so on, I don’t like having to choose a file name that looks cool in the URL, I don’t like having to fix the file extension in order for the web server to send the correct headers. Do you understand what I’m saying? I’m just too old for that.
I created Zetsu to solve this problem, you simply upload the file and forget about it. It gives you a URL, and it doesn’t matter the file name or extension, you don’t have to worry about folders, the URL is immutable which means that even if you change the filename, the URL remains the same. It’s safe against IDOR1 attacks and a win-win for me in every situation. If you want to export the database you can simply download a text file with all URLs and download the files at once using a tool like Wget2, there is no secret about it.
Characteristics
It’s written entirely in TypeScript and strictly follows the Web standards, in other words, it has no NodeJS specific code and can be hosted (theoretically) on any platform. It’s compatible with OpenAPI v3 and makes heavy use of HTTP cache headers, so files can be stored in a CDN like Cloudflare for long periods of time.
The files are identified by a Sha-1 checksum and all requests have an Etag header which makes you and your server save bandwidth.
The only thing I regret doing was choosing SHA-1 as the checksum algorithm, its hexadecimal value is too long for a URL, and now it’s too late to change it. If I were to recreate this project, I would probably choose the xxHash323 in base62 format. This algorithm has enough combinations for storing huge amounts of files and can be short enough to be used for URLs.
Example:
https://example.com/download/27k17JBqV0EL7
It also has a nice little frontend that you can use to manage your files:
Limitations
Cloudflare D1 has a 2-megabyte limit for the table row size4 which means you cannot store files that are bigger than that. Even if you pay for a plan you cannot surpass this limitation.
Due to this fact, the Zetsu API can only be used to host small files, such as images, PDFs, icons, and text files. You cannot use it to store music and videos.
Conclusion
So yeah, it’s sort of a ramble. I just wanted to explain the motivations behind my new little project and I hope you guys like it.
Peace out!