SimGym and Rollouts: How Shopify is bringing real A/B testing into the admin

For years, testing on Shopify has been fragmented. Third-party tools, delayed data, statistical uncertainty, and changes that only reveal their impact after traffic and revenue are already on the line.

SimGym and Rollouts signal a different approach. Shopify is pulling experimentation closer to where decisions are made and earlier in the lifecycle of a change.

Testing before risk exists

Traditional A/B testing assumes you ship first and learn later.

SimGym inverts that model.

Instead of testing on live traffic, SimGym uses AI shoppers to simulate real buying behavior before a change ever reaches customers. These agents browse products, navigate collections, add items to cart, and provide qualitative feedback on friction, clarity, and flow.

The result is not just metrics, but explanation. Why shoppers struggled. Where they hesitated. What prevented progression.

That matters because most storefront mistakes are obvious in hindsight, but expensive in production.

Rollouts close the loop

SimGym answers the question, should we ship this?

Rollouts answers, how and when should we ship it?

Rollouts lets teams schedule theme changes, test variations, and control exposure directly inside the admin. No brittle scripts. No external tooling. No guessing whether a launch window will collide with a campaign or traffic spike.

Together, SimGym and Rollouts turn testing into a continuous practice rather than a risky event.

This is Shopify redefining experimentation

The important shift is not AI shoppers or scheduling alone. It is ownership.

Shopify is moving experimentation into the core platform instead of outsourcing it to plugins and workarounds. Testing becomes a first-class workflow tied to themes, launches, and performance, not a separate analytics exercise.

That changes behavior.

Teams test earlier. Ship with more confidence. And rely less on gut feel when stakes are high.

Where Nexus Commerce fits

SimGym and Rollouts create clarity. Nexus Commerce turns that clarity into execution.

Inside Nexus Commerce, teams can take SimGym insights and Rollout learnings and translate them into structured recommendations, prioritized tasks, and concrete execution plans without leaving an AI-powered workspace.

We help teams:

• Interpret AI shopper feedback and testing results

• Decide what to change next and what to ignore

• Turn learnings into shipped improvements across themes, merchandising, and UX

Shopify is building the testing foundation. Nexus Commerce helps teams operationalize it.

Test smarter. Ship faster. Learn without breaking trust.

Get started for free at nexuscommerce.co