An app sandbox with simulated tools

Why an app needs a sandbox

Most apps lean on outside services. A shop takes payments. A booking app sends a confirmation email. A support tool posts to a team chat. To try any of that for real, you need accounts, keys and approvals before you know if the idea works.

Shipmunk starts without them. When a part of your app needs a customer account or key, the builder plans a stand-in for it. You can start with no accounts at all and add your own keys when you are ready.

How the app sandbox with simulated tools works

Each stand-in is real code with a fake service behind it. The fake keeps state and follows the real service's rules. Your app talks to it through the same interface it would use for the real thing.

• Payments, email, chat and more. The builder knows tools such as Stripe, Gmail, Slack, HubSpot, Zendesk, Google Calendar and GitHub, and can stand in for any other service that needs an account.
• Mail and SMS stay inside. Messages your app sends are recorded by the fake for the tests. They never reach a real inbox or phone.
• Labelled Simulated. On your iteration, each stand-in carries a Simulated label, so you always know which parts are still pretend.
• Lively previews. In the preview, a simulated part answers in the shape the real service would, and keeps what you did, so clicking through feels like the real thing.

Inside the app itself there are no "demo" banners. The app looks and behaves as it would with the real service, which is what you want to judge.

Why this beats a mock that always says yes

The easy fake returns "email sent" and moves on. It never fails, so it proves nothing. Shipmunk's builder is told plainly not to write that kind of fake.

A stand-in that keeps state can be checked. A test can book a class and then ask the fake: is there exactly one confirmation email, to the right person? Was the charge recorded with the right amount? If the app forgot to send the email, or charged the wrong total, the check fails and the builder has to fix it.

How the builder checks its own work

1. Plan the checks

   Before any code is written, the plan lists checks a customer would notice, each with a situation and what should happen. "The app imports" does not count.

2. Build in a fresh container

   The app is built in a fresh Linux container of its own. The build never calls the real services it stands in for.

3. Run every check

   The checks run against the app and its stand-ins. Each check starts on its own empty database, so one test cannot spoil the next. The builder cannot edit the checks to make them pass: they are put back before every run.

4. Fix what fails

   A failing check gets a fix round. A reviewer reads the failure, finds the cause and tells the builder what to change. This repeats within a set number of rounds.

5. Done only when the checks pass

   If the required checks still fail at the end, the iteration says so plainly and names them. If a late change breaks a check that passed earlier, the earlier passing version is kept.

For phone apps there is one more step: the app is opened on a hosted phone. Read more on how Shipmunk builds and tests phone apps, or see the whole flow in how Shipmunk works from idea to tested app.

What you see on an iteration

When an iteration talks to outside tools or has checks to show, it gets a Sandbox card. It shows your app in the middle and a line to each tool it talks to, marked Simulated or Real. Below that is a table of situations: what was tried, what should happen and the result.

A situation that failed in one round and passed after a fix shows as resolved. The card only reads "Proven in its sandbox" when every situation passed.

Making a simulated part real

When the builder asks its questions, you can skip one and let it be simulated for now. Later, on a finished iteration, press Make it real (or Connect on the tool). Tell the builder what the real service uses, and it rebuilds only that part.

Each simulated part switches to the real service as soon as its settings are present, such as the mail server's host and login. Shipmunk works with the names of those settings, never the secret values. Every download includes a setup guide listing the simulated parts and the settings that make each one real. Our security page covers how builds and secrets are handled.

What is live today and what is in closed beta

Live today: stand-ins planned for each build, checks run against them, fix rounds, the Sandbox card and Make it real.

Not live yet:

• Seeding the simulated tools with your own test data, inspecting every change and resetting them.
• A log of every call your app made to a simulated tool during the checks.
• Testing with simulated users who hesitate, mistype, cancel or retry.

Shipmunk itself is in closed beta, and we do not give dates for these.

What the Sandbox is not

The Sandbox is the rehearsal, not production. Passing checks against stand-ins means the app behaves well against services that follow the real rules. It does not mean your real accounts are set up.

Launching still requires live accounts, provider approval and production checks. Shipmunk helps you reach that stage with fewer unknowns. For more answers, see the Shipmunk FAQ.