# Mapshroom — product and shader workflow

Context version: 1.0.0. Updated: 2026-09-18. Compatible with MCP 0.4.0.
Canonical public copy: https://mapshroom.dev/mcp/context.md
MCP resource: mapshroom://docs/context/v1. Tool: get_mapshroom_context (no arguments).
Setup instructions: https://mapshroom.dev/mcp/agents.md

## What Mapshroom does

Mapshroom is a browser workspace for projection mapping on sculptures, plants and irregular surfaces. The user prepares an image, optionally derives depth or masks, aligns the projected image to the physical object, and animates it with GLSL shaders and a timeline. The browser stores the project and renders the result on the device. A depth map helps effects follow apparent shape; it is not a measured 3D scan or automatic projector calibration.

The MCP server supplies this context, the exact shader contract, previews and shader delivery links. The AI in the user's assistant writes the GLSL. The server validates shader structure and control values, serializes data and compresses links. It does not run an LLM, infer depth, render frames on a server GPU, see the user's project or perform mapping alignment. Actual GLSL compilation and rendering happen in the preview/workspace browser. Successful tools/call is not proof of successful GPU compilation.

## Workflow: photo → depth → shader → projection

1. Photograph the object from a viewpoint suitable for aligning the projection; import the photo into Mapshroom. Keep camera, object and projector positions consistent during alignment. See https://mapshroom.dev/tutorial/ for the visual instructions.
2. If the effect needs depth, use Mapshroom's depth-generation interface on the photo, inspect the result and choose a grayscale output. The depth estimator runs on the user's device and may download model weights. A colored depth visualization or a photo/depth overlay is not a grayscale depth input. A mask can restrict the object separately. Depth estimation, masks and image edits are currently user operations in the app, not MCP tools.
3. Select the intended image/depth asset in the workspace. Inspect the near/far convention visually. Estimated depth is relative and can be normalized or contrast-adjusted: do not interpret pixel values as meters. For an imported map, ask which convention it uses or provide an inversion control.
4. Tell the assistant the input kind and desired effect. Read get_mapshroom_context, then get_shader_spec. If the input is unknown, clarify it before writing a depth-dependent effect; for a demonstration, explicitly use the synthetic demo below.
5. Generate and preview the shader on the matching input. The default neutral white texture is suitable for a procedural effect; it cannot demonstrate depth-dependent behavior. In the preview choose Demo depth or load the real image locally. Read compilation errors and adjust controls before accepting the result.
6. Open the returned import link, or use acknowledged local pairing when configured. The link carries only shader ID, name, GLSL and control values. Load/select the same real depth asset in Mapshroom to reproduce the preview. Neither the chosen preview image nor the demo image travels inside the import link. Existing media, mapping and timeline steps remain; the new shader is appended. A new empty workspace receives a white canvas, not an automatically generated depth map.
7. Select the new shader step, align the projection in the workspace and judge it on the object. Input image, resolution, mapping, timing, projector brightness and surface material can change the result. Save the project in Mapshroom. Projects in different browser profiles/devices are separate.

## What the shader receives

One image sampler is passed to `vec4 processColor(sampler2D tex, vec2 uv, float time, vec2 resolution)`. `uv` is normalized image coordinates, `time` is seconds and `resolution` is the current render size. The shader does not receive both a photo and an independent depth texture. There is no automatic inputMode conversion or second depth sampler in this MCP contract.

| Input | What its pixels mean | Appropriate shader behavior |
| --- | --- | --- |
| Photo | RGB colors, with possible alpha | Sample texture(tex, uv); brightness is not inferred geometry. |
| Grayscale depth | Relative depth, usually equal R/G/B | Read .r; document near/far and expose inversion when needed. |
| Binary/soft mask | Membership or opacity | Threshold or weight the effect; it is not a continuous depth map. |
| Colored segmentation | Region labels encoded as colors | Match known labels; do not interpret .r as physical distance. |
| Neutral white | Constant 1 in each channel | Useful for procedural effects that ignore tex; no shape information. |

Keep source alpha when the effect is intended to respect image transparency. Custom float/int sliders, vec3 colors and bool switches need the exact metadata from get_shader_spec. Its instruction to return only GLSL concerns the `code` argument; continue to call the preview/delivery tools and explain the result in the conversation. Use only advertised tools. Do not invent a depth-generation, image-upload, project-read, second-texture or projection-calibration tool.

## Preview and delivery

- `preview_shader`: MCP Apps view plus a browser fallback URL.
- `get_inline_preview`: self-contained HTML for hosts that can actually render HTML/WebGL. Do not claim a visible canvas if the host only displays text.
- Both accept optional `previewInput: "neutral" | "demo-depth"`. This only selects a built-in preview texture. It does not infer depth or transfer media. The image picker decodes files locally without uploading them to MCP; previews downscale large images to at most 960 pixels on the longest side.
- Both previews have two delivery actions: Open on Mapshroom and Copy. Copy includes current control values as GLSL defaults. Preview slider changes are not sent back to the assistant automatically; use the preview's own link/copy action for those values, or explicitly supply them in subsequent tool calls.
- `get_shader_delivery`: returns the compressed import URL and GLSL, without changing any project. Preserve shaderId when retrying the same shader; use a new ID for a changed shader.
- Optional local-only tools: get_workspace_connection, list_workspaces, send_shader_to_workspace. They require the local STDIO server on the same computer as the browser. Never ask a remote-only user for a pairing code. Report local delivery only after acknowledgement.

## Reproducible depth demonstration

Open https://mapshroom.dev/ai/?example=depth-contours for the complete preview.
Download the input at https://mapshroom.dev/mcp/demo-depth.png to reproduce the effect in the workspace. After importing the shader, select its timeline step, choose Load Asset (This Shader Only), Import image, select the PNG and press Use selection. Keep Use step asset as shader input enabled. This assigns the demonstration to that step without replacing the project's other assets. It is an opaque 640 × 360 synthetic grayscale image of two rounded shapes over a sloping background: white is nearer, black farther. It is a deterministic test fixture, not a depth estimate of the user's photo and not a measured distance field.

The PNG and built-in preview texture come from the same generator. The examples below are also returned in get_mapshroom_context. For a first MCP test, take the depth-contours example's code and call preview_shader with previewInput="demo-depth". Then obtain get_shader_delivery using the same code and controls. Inspect the real GPU preview before claiming the shader works on the user's image.

### chromatic-waves

Input: procedural. A procedural effect: it deliberately ignores the image.
Preview: https://mapshroom.dev/ai/?example=chromatic-waves

```glsl
// NAME: Chromatic Waves
// MAPSHROOM PROMPT: "Create slow chromatic waves with speed, scale and intensity controls."
uniform float speed; // @min 0 @max 3 @default 0.6
uniform float scale; // @min 1 @max 12 @default 4
uniform float intensity; // @min 0.1 @max 1 @default 0.8
vec4 processColor(sampler2D tex, vec2 uv, float time, vec2 resolution) {
  vec2 p = (uv - 0.5) * vec2(resolution.x / resolution.y, 1.0);
  float wave = sin(p.x * scale + time * speed) + cos(p.y * scale - time * speed);
  vec3 col = 0.5 + 0.5 * cos(wave + vec3(0.0, 2.0, 4.0));
  return vec4(col * intensity, 1.0);
}
```

### photo-pulse

Input: photo. A photo effect: choose a local photo to see its actual colors and details.
Preview: https://mapshroom.dev/ai/?example=photo-pulse

```glsl
// NAME: Photo Pulse
// MAPSHROOM PROMPT: "Gently pulse my photo with adjustable speed, amount and tint."
uniform float speed; // @min 0 @max 3 @default 0.5
uniform float amount; // @min 0 @max 1 @default 0.4
uniform vec3 tint; // @default 0.4,0.8,1.0
vec4 processColor(sampler2D tex, vec2 uv, float time, vec2 resolution) {
  vec4 source = texture(tex, uv);
  float pulse = 0.5 + 0.5 * sin(time * speed);
  return vec4(source.rgb * mix(vec3(1.0), tint, amount * pulse), source.a);
}
```

### depth-contours

Input: grayscale-depth. Animated contours follow depth values. White means near in the synthetic sample; invertDepth handles the opposite convention.
Preview: https://mapshroom.dev/ai/?example=depth-contours

```glsl
// NAME: Depth Contours
// MAPSHROOM PROMPT: "Animate luminous contours on a grayscale depth map with speed, bands, width, tint and depth inversion controls."
uniform float speed; // @min 0 @max 2 @default 0.2
uniform float bands; // @min 2 @max 24 @default 10
uniform float width; // @min 0.02 @max 0.3 @default 0.1
uniform vec3 tint; // @default 0.3,1.0,0.7
uniform bool invertDepth; // @default false
vec4 processColor(sampler2D tex, vec2 uv, float time, vec2 resolution) {
  vec4 source = texture(tex, uv);
  float depth = clamp(source.r, 0.0, 1.0);
  if (invertDepth) depth = 1.0 - depth;
  float phase = fract(depth * bands - time * speed);
  float distanceToLine = min(phase, 1.0 - phase);
  float line = 1.0 - smoothstep(width * 0.5, width, distanceToLine);
  vec3 color = tint * (0.08 + 0.18 * depth + 0.74 * line);
  return vec4(color, source.a);
}
```

## Limits and recovery

Shader source is limited to 32 KiB UTF-8; at most 32 controls and 64 KiB per HTTP request. Keep effects simple enough for the user's GPU. The server checks format and values; it cannot guarantee frame rate or visually correct results. Browser compile errors are visible to the user, not automatically returned to the AI. A shader designed for two independent textures requires a future rendering contract or an explicitly prepared packed input, not an invented sampler.

Remote endpoint: https://mapshroom.dev/api/mcp, Streamable HTTP, no authentication. The remote server exposes five tools; the optional local server exposes eight. Refresh the MCP connection to discover new tools. Resources are made available to the host; their presence alone does not mean the model has read them, which is why the context is also a tool. Search indexing never installs the MCP connector.

For a blank/flat depth effect, check that the selected input is a varied grayscale depth map, not the default white texture or a colorized depth preview. For a compile error, copy the browser error back to the assistant. For a reversed effect, toggle invertDepth. For a different workspace result, check the input asset, control values and active timeline step. If the host cannot render a preview, open the browser fallback. The remote server is subject to hosting and rate limits; retry 429 after Retry-After or use the optional local server.