FAQ

I can be an extension of your team, available to help with decision making and planning around AI. I can help with building some tools, but just as often it will be better that we use an existing solution or hire someone else to build something, and I will be there help you decide. Either way, it will start with an initial meeting or video call to discuss your needs and goals, a small project (less than 10 hours), then an onging engagement of either a half or full day every two weeks that can be used for planning, training, building, etc.

No lock in. At any time you can decide that you no longer need my help and you can end the engagement with no advance notice required.

I use a variety of industry-leading AI tools including n8n, OpenAI, Anthropic and ElevenLabs. If we need to use a tool that I am not experienced with, any time learning the tool will be at my expense, not yours.

Yes, this website was built by AI, although with quite a bit of oversight by me. The level of human oversight required is dropping rapidly, but it still helps to be able to to code to build a decent website.

No, but it will involve some learning as the goal is for you to become self-sufficient, not reliant on me.

No, provided you take the time to learn the tools, you will be self-sufficient and able to maintain and improve the systems yourself.

Contact

Email: marc [at] marcharrison.net

LinkedIn: https://www.linkedin.com/in/marcharrison/

Location: Melbourne, Australia

A Lesson on the risks of AI chatbots

▶ Version Details

Summary:

This article was written by me, with a final spelling and grammar check conducted by AI.

Two days ago, users of the popular AI coding tool Cursor found themselves being logged out of one device when they opened the app on another. While some software products (e.g., Microsoft Office, Netflix) do enforce this kind of single-device restriction to encourage multiple subscriptions, it’s generally not a popular feature.

This unexpected behavior led many users to contact the company, unsure whether it was a bug or a change in policy. Given that Cursor had made a few unpopular decisions recently, emotions were already high—so when the support team responded by confirming that this was a deliberate policy change, many users were furious.

One problem: it wasn’t a policy change. It was a bug.

The support responses had been generated by AI, which completely made up (or “hallucinated”) the idea that this was an intentional decision. This led to backlash on Reddit and a wave of cancelled subscriptions.

A valuable lesson

The key logic behind AI chatbots isn’t traditional code—it’s human-readable instructions. But AI makes mistakes. And one of the most common? Not following the instructions it’s been given.

We’ll probably never know whether this particular failure was due to poor prompt design or the AI simply ignoring its instructions. Either way, it’s a valuable case study.

A guess at what might have gone wrong

AI chatbots tend to err on the side of agreeing with the user rather than challenging them. (Though not always.) It’s generally too easy to convince an AI that it is wrong.

So it’s easy to imagine the Cursor team previously found that their AI support bot kept agreeing with users who reported bugs—even when those bugs were actually user errors. That would make the company look bad and wouldn’t help users find a quick solution to their problems.

In response, they may have adjusted the system prompt to instruct the AI not to assume everything was a bug. Perhaps they told it to challenge users more often. But if that wording went too far, the bot might have swung in the other direction by refusing to acknowledge even legitimate bugs.

Alternatively, maybe they gave the AI a list of known bugs to help it respond more accurately. But this issue was so new, if there was such a list then this bug might not have been added yet.

In either case, if the AI didn’t know about the bug but was trying to be helpful (instead of saying “I don’t know”), it’s easy to see how it might guess that this was a deliberate policy based on it’s training data that included plenty of examples of other software companies doing similar things. Following its instructions to support company decisions, the AI confidently gave an incorrect response that lead to the social media backlash.

A few takeaways:

  • Using chatbots for important customer interactions is risky.
  • Not disclosing that the support agent is an AI means most people will assume they’re talking to a human, and that a confident response means that human knows what they’re talking about.
  • Clearly disclosing that the response is AI-generated helps mitigate this risk, as users can then decide whether to trust the information or seek human confirmation.
  • People are rapidly getting used to dealing with AI (many already prefer it over humans for certain tasks) so being open about AI usage is likely to build trust, not erode it.

Tech companies are leading the way in reaping the enormous productivity benefits of AI, but they’re also learning the hard lessons of what can happen when AI is relied on too heavily or isn’t properly configured. Learning from both their wins and their stumbles is a great way for others to be fast followers and to gain the benefits of AI without taking on all the risk of operating right at the bleeding edge.

Read more: