What AI actually does in everyday tools
Artificial intelligence is a broad label for software that detects patterns, classifies information, predicts likely outcomes or generates new material. A phone camera can use machine learning to recognise a scene; a navigation app can estimate traffic; a writing assistant can produce a paragraph from a prompt. Those systems have different purposes and different failure modes, even when they are all marketed as AI.
Generative AI produces text, images, audio, video or code by calculating a plausible continuation from its inputs and learned patterns. It does not check a statement against reality before presenting it. When a prompt asks for a summary of a long document, the tool may omit a qualification. When asked about a niche Aotearoa place, it may substitute a familiar overseas example. A polished interface can make these errors harder to notice.
A transcription service is designed to turn speech into text; a general chatbot is not a guaranteed substitute for a translator, solicitor, doctor, teacher or local cultural adviser. Read the product’s description and privacy settings before treating two tools as interchangeable.
Match the task to the risk
A low-consequence task leaves room to experiment. Brainstorming headings, turning a personal set of notes into a checklist or creating a first draft for private editing can save time. A person still needs to remove invented details, restore missing qualifications and edit the wording for its intended audience.
Risk rises when an output affects another person’s money, health, education, employment, legal position, reputation or access to a service. A tool that sorts job applications, flags a student, recommends a benefit decision or answers a customer complaint is making a consequential contribution, even if a staff member clicks the final button. A human check needs enough time, context and authority to reject the output.
| Task | Suitable role for AI | Human check that matters |
|---|---|---|
| Meeting notes | Create a draft transcript or action list from approved material. | Correct names, commitments, numbers and anything attributed to a speaker. |
| Customer reply | Suggest a calm first draft using a fictional or de-identified example. | Confirm policy, facts, tone and whether the reply reveals private details. |
| Document summary | Group information from documents supplied for the task. | Compare every important claim with the original wording and date. |
| Decision about a person | At most, help organise information for a qualified reviewer. | Check fairness, context, legal duties and the person’s ability to challenge an error. |
| Creative draft | Offer options, variations or production assistance. | Review originality, permission, cultural context and the creator’s own contribution. |
Digital neural models and computational language interfaces in daily office use.
Prompt privacy and sensitive information
A prompt can contain more than the words typed into a box. A document may include names in its text, comments, tracked changes, account numbers, location details or an image that identifies a person. A “clean” summary can still be personal information if a reader could work out who it concerns. Removing the name alone is not always enough.
Before staff use a service, an organisation should know what information enters it, where the provider processes it, how long it keeps prompts and outputs, whether it uses them to improve a model, who can access them and how deletion works. A privacy impact assessment makes those questions concrete. The New Zealand Privacy Commissioner also expects agencies to consider necessity, transparency, Māori engagement, accuracy, access and human review when AI handles personal information.
- Use a business-approved account for work rather than a personal free account with unknown settings.
- Replace names and identifiers with neutral labels before testing a workflow.
- Do not paste medical, banking, employment, student or client records into a public chatbot.
- Check whether uploaded files remain stored after a conversation is deleted.
- Tell people in plain language when automation materially shapes a service they receive.
- Keep a route for a person to correct information or ask for a human review.
An organisation operating in New Zealand may have Privacy Act duties even when its service provider is based elsewhere. The provider’s location does not remove the organisation’s responsibility for the way information is collected and used.
Accuracy depends on the job and the local context
- Recognise model limits
Generative tools can state a wrong answer in polished, confident language. They may invent a case citation, combine two real events, offer an outdated product setting or give an answer that reflects another country’s rules. A citation generated by a chatbot is only a lead until the cited document has been opened and the relevant passage checked.
- Verify factual claims
For a factual draft, break the output into claims that can be checked: a date, a legal rule, a price, a name or an attribution. Verify those against the primary source responsible for the information. If the answer is about a person or a recent change, check the latest record instead of relying on a model’s general knowledge. Put a clear date beside any information that could change.
- Review local meaning
Te reo Māori words, iwi names, whakapapa, mātauranga Māori and references to taonga are not interchangeable decoration. A model may reproduce a spelling while misunderstanding the relationship behind it. If a project concerns Māori communities or information, involve people with relevant authority and knowledge while decisions are still open, rather than asking them to approve a finished automated output.
AI can widen access when a person remains in control
Speech-to-text, live captions, translation drafts and text simplification can help people take part in work, study and public services. A student may use transcription to review a lecture; a person with limited hand movement may dictate a message; a community group may prepare a first translation before a fluent speaker checks it. These uses work best when the person can choose the tool, correct the output and keep an alternative route available.
Accessibility features need testing with the people who will use them. Automatic captions can mishear Māori names, place names and speakers who use a Pacific language or a regional accent. A translation can flatten a concept that has no direct English equivalent. A generated plain-language version may omit a condition that matters to someone applying for a service. Reviewers need relevant language and access knowledge; a general proofread alone may miss those issues.
For a public-facing service, explain whether AI is producing captions, routing a request or writing an answer. Make it possible to reach a person when the output is confusing or a user cannot complete the task. An accessibility feature needs to let people complete the task on equal terms.
Creative work, training material and permission
AI can help a writer explore structure or a designer produce rough concepts, but generated material is not automatically accurate, original or free of rights concerns. A user may not know which examples shaped an output, and similar prompts can lead different users toward similar results. The sources and permissions need checking before the work is sold, licensed, submitted for assessment or credited to a named creator.
Separate assistance from authorship. Keep drafts and records that show which parts a person created, which were machine-generated, which source files were supplied and what revisions were made. Check the terms of the tool and the rules of the publisher, client, school or competition. Do not upload someone else’s unpublished manuscript, artwork, song stems or photographs unless the owner has authorised that use.
A workplace AI policy should name allowed tasks, prohibited inputs, approved tools, review responsibility and the disclosure standard for external work. A short set of examples is easier to follow than a blanket sentence saying “use AI responsibly.” The rules should be updated as products, contractual terms and privacy settings change.
A verifiable AI workflow
- Write the job in one sentence. Define the result, audience and limits before opening a tool.
- Choose low-risk material. Use public sources, fictional examples or de-identified notes unless an approved system has been assessed for sensitive use.
- Ask for a bounded output. Specify the format and say what the system must leave uncertain rather than guess.
- Check the source trail. Open original documents for dates, numbers, quotes and local rules.
- Revise for people. Restore missing context, check accessibility and make the wording fit its actual audience.
- Keep responsibility visible. Record who approved a consequential decision and how an affected person can challenge it.
A community group can use an approved tool to turn a public venue page into a draft event checklist. A volunteer still checks the opening hours, accessibility details and emergency arrangements with the venue. The tool handles the draft’s formatting; the venue supplies the information used to check it.
Practical questions about AI
Can a business use AI with personal information?
It can only do so in a way that meets its privacy obligations. The business needs to understand the data flow, the provider’s handling and retention terms, the purpose for collecting the information, and how accuracy, access, security and transparency will be managed. A privacy impact assessment helps expose gaps before a service goes live.
Can I rely on a generated answer without checking it?
Not on its own. Check each material claim with a current primary source, especially law, health advice, dates, figures and information about an identifiable person. If no reliable source supports a sentence, remove it or state the uncertainty clearly.
Should an organisation ban AI outright?
A blanket rule can be easy to explain, but it may also hide informal use. A task-based policy can distinguish practice, accessibility support and low-risk drafting from confidential records, graded work, hiring and automated decisions. The right boundary depends on the purpose, age group, data and ability to review errors.
How can we tell whether an AI workflow is helping?
It improves the whole task after review: fewer repetitive steps, clearer information or a more accessible first draft without adding extra correction work or exposing data. If the reviewer must rebuild the output from scratch, the workflow has shifted effort instead of reducing it.