How to build a digital commissioning test plan

The twin looked impressive in demo mode. Then the customer asked what had actually been verified, and the answer was a spreadsheet, three engineers' memory, and a list of alarms that "usually clear themselves." A digital commissioning test plan exists to prevent that moment.

It is the documented basis for what you test in virtual commissioning, how you know a scenario passed, and what remains open. Each entry has a number, a trigger, expected behaviour, pass criteria, an owner, and a status. It is more than rows in Excel: it is the shared language between automation, mechanics, project management, and quality.

The plan connects three things: PLC code (or soft-PLC), the digital model, and the people who must agree on results. Without that link, virtual runs become ad hoc demonstrations where the automation engineer knows what works, but nobody else can trace it to acceptance.

The concept overlaps with a FAT protocol, but the job is different. The test plan is the living workbook during development. The FAT protocol is often customer-facing documentation tied to formal sign-off. Many FAT test points start as virtual scenarios that already ran and logged weeks earlier.

A strong plan is readable by someone who did not write the PLC. Triggers, steps, expected results, and evidence requirements should be written in project language, not only as internal tag names. When a new engineer joins or the customer joins a preview session, the scenario should run without a verbal handover.

Most teams already have a model and PLC simulation access. Failure starts when nobody agrees what must be proved. The demo tour shows normal operation beautifully, but e-stop mid-cycle, format changeover, and an empty buffer because the previous station stopped never become named test cases.

Without fixed scenario names and numbers, quality depends on who is in the room that day. A senior automation engineer remembers the critical interlocks; a newer colleague tests the happy path. At physical FAT, the customer finds gaps you thought were covered because "we ran the simulation."

Open issues scatter across email, chat, and meeting notes. Nobody can answer which PLC version proved scenario VC-04, whether the defect is closed, or if the customer must see it again at the physical machine. Without a single register, virtual testing never enters the quality system.

Mechanical and automation teams often work from different mental checklists. Mechanics think clearance, motion, and operator access; automation thinks state machines and alarm text. A test plan forces both views into one structure so a changeover is not only "recipe loaded" in the PLC but meaningful in the model and on the HMI.

Each scenario should describe at least five things: how it starts (trigger), who does what (roles), what to observe (expected behaviour), when it passes (criteria), and how it is recorded (evidence). Add a sixth line on twin limitations when the model simplifies sensors, timing, or material.

Triggers must be specific. "Test e-stop" is not enough: state the cycle phase, the station, and whether the operator must act on the HMI. "Start auto with recipe A after full reset" is a trigger you can rerun next month.

Pass criteria must be observable. "Sequence works" is not a criterion. "All motion stops within the defined time; HMI shows safety stop; restart follows the documented reset sequence" is testable for an internal reviewer or a customer observer.

Assign an owner per scenario: automation, safety, mechanics, or project management. Ownership means who closes open points, not only who pressed start in simulation. When results are ambiguous, a named decision-maker must log the call.

Version the test list with the PLC project. When logic changes, update scenarios or mark them obsolete. Do not carry green results from an old PLC version into new FAT documentation without matching version numbers.

In our case: digital commissioning test plan, we show how one machine-line team structured a test list, acceptance matrix, and issue tracking around a Unity-based twin with Siemens and Beckhoff PLC interfaces.

Start with what hurts if it fails late: safety interlocks and e-stop recovery, mode changes between manual, auto, and service, format or recipe changeover, start and stop under partial load, and signal exchange with equipment before and after your machine when it is in scope.

Use your own history. Which five to ten problems appeared at your last three FAT events? If the answer involves logic, HMI, or sequence behaviour, it likely belongs in the virtual column first.

On smaller projects with few sequences and a small team, five well-written scenarios are often enough. One page with a name, how to start the test, and when it passes beats an empty spreadsheet, and you can always expand the list once the structure proves its value.

Prioritize negative tests: wrong recipe selected, guard opened mid-cycle, sensor loss, MES timeout, conveyor blockage. Virtual environments make these repeatable without scrap, noise, or safety exposure on the floor.

Operator flow is often under-tested. Correct PLC logic with confusing alarm text or reset order still fails in production. Include scenarios where the operator must acknowledge, escalate, or choose between allowed HMI actions under pressure.

A useful split between virtual and physical scope is in our virtual FAT vs. physical FAT guide. Mark each new scenario as virtual-only, physical-only, or both before you expand the test list beyond the first ten scenarios.

Write criteria you can carry from virtual runs to physical FAT without rewriting. Base them on observable behaviour, not implementation detail. "Valve Q12 energizes" is internal. "Product releases from station 2 within agreed cycle time under recipe A" is a criterion the buyer can relate to when X is in the spec.

Separate must-pass from nice-to-have. Must-pass scenarios block the next milestone when they fail. Nice-to-have can move to SAT or a later software drop but must still be logged so they do not vanish.

For safety, reference your risk assessment and PLd or SIL requirements where relevant. Virtual pass often proves interlock logic; physical FAT confirms distances, door switches, and measured stop times. State in the plan what virtual green covers and what still needs confirmation at the physical machine with the customer present.

Use a consistent severity scale for open points: critical (fix before customer test), major (fix or agreed workaround), minor (documented, may follow). Without a scale, every defect becomes "we will take it later."

When the customer joins a virtual preview, walk through pass criteria for the scenarios you show. That reduces arguments at physical FAT about what "passed" meant in simulation.

Start with a fixed agenda: which scenarios run (e.g. VC-01 to VC-05), which PLC version is in scope, and which open points from the last session must close or reopen. Without an agenda, the meeting becomes a free demonstration.

Roles matter. A facilitator keeps tempo; the automation engineer drives logic and simulation; the HMI owner watches operator flow; project management logs decisions and scope. Mechanics join when motion, clearance, or changeover is central. Safety weighs in on interlock and reset scenarios.

Run scenarios in priority order, not random order. Log deviations with scenario number, actual behaviour, expected behaviour, severity, and owner. Capture screenshots or short recordings when reproduction is unclear.

Close with visible status: green, amber, or red per critical scenario, and what must happen before the next session. Share the protocol the same day. A virtual review without a protocol does not count in the customer's acceptance story.

Plan at least one internal dry run before inviting the customer. The first time simulation starts should not be in front of them. Internal time is cheaper than calendar and trust.

The strongest return on a digital test plan is reuse. Scenarios that passed virtually become checklist rows at physical FAT. Scenarios that failed virtually and were fixed should rerun as regression after every major PLC change.

Mark three columns in the test list: virtual only, physical only, both. That stops virtual green from being over-read and stops you forgetting physical tests for items the twin cannot represent.

Operator training can use the same scenarios at a slower pace: alarm, reset, changeover, escalation. When training modules share scenario numbers with the test plan, you know which flows operators practiced before the machine ships.

Service inherits value later. Site fault reports can map back to a scenario: "VC-07 failed again under similar buffer conditions." That speeds root cause compared to unstructured debugging.

When you need to justify the effort to project sponsors, the virtual commissioning ROI calculator models engineering, FAT, and SAT separately so you can build a project-specific case, not a generic promise.

Update the plan after FAT. Physical findings that were not modelled become new scenarios or notes on twin limits. The document stays true for the next variant or the next line in the same product family.