8 Comments

I see there are many ulid libraries available. Which one are you using? Thanks in advance.

Expand full comment
author

The one linked in the article

Expand full comment

OMG, I didn't see the link. Thanks for the quick replay! :-)

Expand full comment

I’d add the requests library in there too.

Expand full comment
May 19, 2023·edited May 19, 2023

Can you use diskcache as a message queue like in RabbitmMQ or a STREAM like in Redis? Also, can I use it with asynchronous code?

Expand full comment
author
May 19, 2023·edited May 19, 2023Author

discache is only a key/value store, is has none of the richer redis features, and since it doesn't connect to the network, it cant act as a message queue.

That being said, it does come with a deque implementation, which mean you can manually turn it into a queue: https://grantjenks.com/docs/diskcache/tutorial.html#deque. This, coupled with transactions, mean you could technically turn it into a poor man task queue.

As for asynchronous code, asyncio doesn't work with the filesystem, and diskcache doesn't touch the network, only files. Given the speed of the request, it's usually doesn't matter and you can use it as-is. But if you ever reach a point where it does, the typical way of taking care of the problem is putting fs accesses in asyncio.run_in_executor.

Expand full comment
May 19, 2023·edited May 19, 2023

Thanks! I was sort of hoping that I could fetch data from an external REST API, dump it into diskcache, and also pop it for further processing, all in one process.

Expand full comment
author

That's a very good use case for it.

Expand full comment