Tecman Talks AI Field Notes: How we took the admin out of solution testing
Reading time: 3 - 6 minutes
Welcome to Field Notes, a new Tecman Talks AI blog spin-off sharing our lived and breathed experiences of experimenting with AI across our own business.
There's plenty of noise around AI, but this blog series will cut straight to what matters: honest accounts of where we've used it, what we've learnt and – most importantly – why it matters for our customers.
In this very first edition, we kick things off by putting testing under the spotlight.
The challenge: making sure everything works as needed
Before any Microsoft Dynamics 365 Business Central solution goes live, we need to be confident it works exactly as our customers expect. That means testing every process, from creating sales orders and purchasing stock, through to forecasting, invoicing and reporting.
For every project, our consultants create a detailed solution testing document – an ultimate guide to testing the solution. This helps us make sure every business process, custom development and third-party add-on has been checked before go-live.
Creating these testing documents can be a long and painstaking process that involves:
- Reviewing the Business Requirements Document that explains how the customer’s business should work
- Reading through the development backlog (in our internal system for managing all development) to see what customisations have been built and what changes have been made throughout the project
- Comparing the two, identifying everything that needs testing and creating a complete testing pack
Depending on project scope, this process can take several days to complete.
Where did AI come in?
On a recent Business Central implementation for a commercial ventilation product manufacturer, one of our consultants wanted to see if they could reduce the time needed to create the testing pack, without compromising quality.
Rather than manually comparing project documents and development records, he used AI to bring the information together from DevOps (the Microsoft software we use to plan, build, test and ship all our custom development on) more efficiently.
The Claude and DevOps agent reviewed:
- The Business Requirements Document
- The Azure DevOps backlog containing the project's development work
- A reusable AI skill (think of this as the instructions for the agent) developed by our team to guide the process
The AI analysed the information it surfaced and created a structured solution testing document – in a matter of hours.
But the benefit wasn't just the time saved. The AI-made testing pack traced every test back to the original requirement that prompted it, helping make sure nothing important was missed.
What was the AI actually doing?
One of the biggest misconceptions about AI is that it replaces expertise. That's not what happened here.
The AI didn't perform the testing. It didn't decide whether a process was correct. And it didn't replace the consultant. Instead, it handled the administrative work that sits behind testing.
Typically, a consultant would spend hours moving between project documents, crosschecking and drawing up the testing pack. The AI completed a lot of that groundwork automatically, comparing information, organising it into a structured testing framework and highlighting how each test related to the original requirement.
The consultant then reviewed the output, applied their project knowledge and refined the final document.
The expertise stayed firmly with the consultant. The repetitive work didn't.
Why this matters for customers
While this process happens behind the scenes, the benefits are felt throughout the project lifecycle.
Saving time is useful, but only if that time is spent on something more valuable. In this case, it means consultants spend less time on administration and more time helping customers.
Instead of manually building documents, they can focus on process design, user training, testing support and preparing teams for go-live.
The benefits continue beyond implementation, too. Because the testing document clearly links business processes, requirements and system changes, it becomes much easier for someone new to understand how the solution works.
This improves project handovers and helps support teams investigate issues more efficiently. Plus, when customers request future enhancements, it becomes quicker to understand what existing developments might be affected and provides a consistent approach. Consistency might not sound exciting, but it becomes very important when you're reviewing testing results, preparing for go-live or handing a solution over to support.
The result is not just a faster internal process; it's a better customer experience.
Could the same approach work in your business?
The interesting thing about this example is that it isn’t unique to testing – it's about bringing information together.
If your business has processes that involve manual document comparison, cross-checking information across multiple systems or making sure different records match, AI could help lessen that administrative burden. Think…
- Comparing customer requirements with completed project work
- Checking quality procedures against operational processes
- Reviewing supplier contracts against purchasing activity
- Comparing forecasts against actual sales or production performance
What we've learned
The biggest lesson from this project wasn't the testing document itself, but the process behind it.
Once we understood how AI could support this type of work, we could save the approach and reuse it on future projects. That's where the real value of AI comes from. Not one-off experiments, but repeatable ways of working that improve quality and save time.
And don’t forget, AI is only as good as the information it's given. This process worked well because the project documentation was detailed and the development records were well maintained. Good AI still relies on good data and experienced people – in this case, our consultant.
Looking ahead
Ready to see where AI can fit in your business solutions? Get in touch today.