What we’ve learned using always-on AI assistants

Over the past few weeks, we’ve been trying personal AI assistants that work in the background, with access to tools we use to run MarsBased. We’ve used Dot, from ChatGPT, for company work and tried Instinct separately for personal tasks.

Our experience so far is with management and operations. We haven’t tested these assistants enough for software development to have an opinion yet.

Why we went with Dot

We chose Dot mainly because of how it fits into our existing company setup. While most of the team uses Claude for their work, at the management level, we also have ChatGPT to compare both of them, and because it works better with documents and certain integrations like Google Docs.

We access it through our ChatGPT business subscription, using the connectors we’ve already enabled for the company. That made it easier to assess whether we were comfortable giving it access to our work.

We weren’t as comfortable using Instinct with company information, especially not now that we’re being audited for the ISO 27001 certification. We didn’t have a clear enough picture of its business offering or how our data would be handled, so we kept our tests personal.

That matters when you’re connecting an assistant to email, internal conversations and documents that may contain client information. Before trying to find useful tasks for it, we needed to be comfortable with what it could access and under which conditions.

It caught things we’d missed

The most useful thing Dot has done so far is flag problems we hadn’t asked it to look for.

During an application audit, the person carrying it out mixed up the app name and started auditing a different application. Dot picked this up from a Slack thread where we weren’t even mentioned. It checked the conversation against the project and audit documentation, spotted the mismatch, and alerted us in time to correct it.

Another example: our Google Workspace payment failed because the credit card was declined. The notification arrived by email, but nobody noticed it at the time. Dot flagged it.

In both cases, the information was already there. We’d missed the payment email, and we weren’t following the Slack thread about the audit. Dot brought both to our attention without us having to ask.

We already spend plenty of time checking email, reading Slack and keeping up with projects. We still miss things. Having an assistant pick up on a connection between a conversation and a document has proved useful in a way that’s difficult to get from a chat you only open when you have a question.

More than a scheduled task

Our CTO, Xavier wrote about using Claude Cowork for daily briefings. Scheduled tasks are useful for that kind of work: you define a recurring job and choose when it should run. Every morning, for example, it can review your calendar and prepare a briefing. The contents change, but the instruction and schedule stay the same.

What interests us about personal assistants is being able to give them instructions that apply whenever something happens, throughout the day. If an email of a particular type arrives, take a specific action. Keep an eye on a conversation and tell us when something needs our attention.

The assistant can work around the clock, checking the sources it has access to. An incoming email, a change in a conversation or a discrepancy it spots can give it a reason to act, without waiting for the next scheduled briefing.

The audit example is what made this difference clear to us. We hadn’t scheduled a check for application names or asked Dot to review that Slack thread. It spotted something that didn’t match the project documentation and brought it up.

Sending work and picking it up later

We also use Dot differently from a normal ChatGPT conversation. We send it tasks throughout the day and pick up the results later. There’s less need to stay in the conversation while it works.

That makes it easier to hand something over when it comes to mind. We don’t always want to stop what we’re doing, open a new session and work through a task with an AI. Sometimes we just want to ask for something and hear back when it’s done.

There’s plenty of routine work we can delegate this way: reviewing LinkedIn connection requests and accepting the ones that fit our criteria, archiving emails that are clearly spam, or finding a gap in the calendar and arranging a meeting. We can also ask it to summarise the notes from meetings we couldn’t attend.

These are tasks we might give a human assistant. We explain what we want, give it the criteria to work with, and let it get on with the job. For LinkedIn requests, for example, that means deciding what makes a connection relevant to us before asking it to accept anyone.

Some of these jobs can be handled as they come in. Others need a bit of back-and-forth. Arranging a meeting might mean finding a suitable time, proposing it and waiting for a reply before sending the invitation. We want to be able to hand over that whole task without having to prompt each step ourselves.

What we still need to try

We’ve mostly handed over tasks as they’ve occurred to us. We still need to spend some time looking at the recurring work we do each week and deciding which parts are worth delegating.

So far, though, the most useful part has been getting a heads-up about things we might otherwise have missed. We didn’t ask Dot to check whether someone was auditing the right application. It noticed the mismatch while following the work.

That’s given us a reason to keep using it while we figure out what else to hand over.