Hi all!

Thanks for volunteering to be a part of Sourcery.info, the project that’ll make it safe to use AI with your documents. I typically work on Sourcery on Mondays, so from now on I’m going to write a short update on Mondays on how things are progressing, what help I need, and to solicit your feedback and opinions. It’ll also help keep me working at pace - I don’t want to get to the end of Monday and have nothing new to say!

I’ve broken ground on development. I started with some mockups, and have now moved on to some simple user interfaces, as well as experimenting with AI models. As soon as I have a working prototype, I will get it into your hands to try! I’m hoping this’ll happen in the next six weeks.

I’ve been running the AI models on my Macbook Pro M1 with 16GB of RAM, and the performance isn’t great, especially for a half-capable model like Llama2. It takes an age to index large documents, and the performance while chatting to them isn’t what I’d consider acceptable at this point.

I do have a plan though - today I experimented with doing some of the heavy lifting on a cloud computer, and even on the lowest tier computer it was significantly faster. I’ve rearchitected Sourcery so that this is an option for you. If you’re dealing with less-sensitive documents, like court reports or government documents, use the cloud solution and save yourself a bunch of time and expense. It’ll still be relatively secure and significantly more private than the publicly available AI systems from OpenAI, Microsoft, Google etc.

If you’re working on some kind of Snowden-leaks or Panama Papers, and everything needs to be super-secure, then you can still run everything securely and locally, but you might need to purchase a beefy machine for the task.

The good news is that the cost of a very AI-capable machine is coming down quickly, and PCs are getting much faster at AI. I predict that by the end of this year, a new mid-tier machine will be quite able to run a decent model at an acceptable speed.

In light of this, I have a few questions for you: How much, if at all, do you deal with really sensitive documents? How much budget would you be able to allocate if you needed a seriously secure environment? What are the specs of your current every-day machine?

Until next week Monday!