Making Interactive Web Apps Agent-Friendly (It Kinda Sucked)

· By Natalie Walls

Making Interactive Web Apps Agent-Friendly (It Kinda Sucked)

The Plan Was Simple

I built a motif pattern generator with a beautiful interactive UI. Then I thought: "Let's make it usable by AI agents!" How hard could it be?

Add a URL-based API:

  1. Encode config as base64
  2. Pass via URL parameter
  3. Agent opens URL, downloads result
  4. Done!

Narrator: It was not done.

What Actually Happened

Round 1: The Emoji Nightmare

The Bug: Agent-generated URLs worked perfectly when I tested them in Node, but failed in the browser with 126 control characters.

The Culprit: Multi-byte UTF-8 emoji encoding. The swimming pool emoji (🏊) is 3 bytes. The beach umbrella (🏖️) has a variation selector (extra byte). When base64-encoded improperly, these turn into garbage.

The Fix: Explicit TextEncoder/TextDecoder usage:

// ❌ WRONG - corrupts multi-byte characters
const base64 = btoa(JSON.stringify(config));

// ✅ CORRECT - preserves UTF-8
const json = JSON.stringify(config);
const encoder = new TextEncoder();
const bytes = encoder.encode(json);
const base64 = Buffer.from(bytes).toString('base64');

The Lesson: If your config has emojis, you MUST use TextEncoder. Or better yet...

Round 2: Just Use ASCII

After hours of debugging emoji encoding, I had an epiphany: why use emojis at all?

The pattern generator supports Lucide icons (simple ASCII strings like "heart", "star"). No multi-byte encoding. No variation selectors. No skin tone modifiers. Just works.

Updated recommendation in SKILL.md:

// ✅ RECOMMENDED for agents
{ type: 'lucide', lucideName: 'heart' }

// ❌ AVOID unless necessary  
{ type: 'emoji', emojiChar: '🏊' }

Agents now have 1,900+ Lucide icons that "just work" without encoding headaches.

Round 3: React State Is Asynchronous (Obviously)

The Bug: URL config loaded successfully, but the pattern showed defaults instead of the configured icons.

The Culprit: Calling generatePattern() immediately after setState():

// ❌ State not updated yet!
setConfig(urlData.config);
handleGenerate(); // Uses OLD config

The Fix: Store URL config in a ref and wait for state updates:

const urlConfigRef = useRef(null);

// Load from URL
urlConfigRef.current = urlData;
setConfig(urlData.config);

// Separate effect watches for state changes
useEffect(() => {
  if (urlConfigRef.current && config.icons.length > 0) {
    generatePattern(urlConfigRef.current.config); // Use ref!
    urlConfigRef.current = null;
  }
}, [config.icons]);

The Lesson: React state updates are async. If you need synchronous access, use a ref.

Round 4: Export vs Preview Divergence

The Bug: Live canvas preview showed no icons, but exported PNG had all 95 icons rendered perfectly.

The Culprit: Lucide icons need to be fetched from CDN. Canvas component wasn't waiting for them, but export function was.

The Fix: Fetch icons during export if needed:

if (source.type === 'lucide' && !source.lucideDataUrl) {
  const { fetchLucideIcon } = await import('./lucide-client');
  dataUrl = await fetchLucideIcon(source.lucideName);
}

The Lesson: Export and preview are separate render paths. Test both!

Round 5: Agents Don't Read Documentation

Initial SKILL.md said "open this URL and download the result." Agents interpreted this as "ask the user to open the URL."

Updated to be explicit:

## Agent Workflow (Complete Automation)

**Agents should automate the entire flow, not ask users to open links!**

1. Build config
2. Encode to base64  
3. Use browser automation to open URL
4. Wait for completion signal
5. Extract blob and send to user

DO NOT: Ask user to "please open this link"

Agents need step-by-step instructions with code examples. Assume nothing.

Round 6: Completion Signals Need Details

Agents were polling for data-pattern-status="complete" but hanging on errors.

Added error hints:

if (error.includes('control character')) {
  document.body.dataset.errorHint = 
    'Config encoding error. Use TextEncoder with UTF-8';
}

Now agents can tell users why it failed, not just "it didn't work."

Round 7: File Extensions Are Important

Downloaded file had no extension. Browser saved as pattern instead of pattern.png.

The fix was correct all along:

downloadFile(blob, `pattern.${format}`); // Already had extension!

But agents weren't seeing the download happen. Added explicit logging:

console.log('Triggering download:', filename, 'Size:', blob.size);

Now agents (and humans debugging) can see exactly what's happening.

What I Learned

1. Multi-byte encoding is a minefield

Emojis, special characters, anything beyond ASCII — they ALL require explicit UTF-8 handling. TextEncoder/TextDecoder is not optional.

Or just avoid them entirely if you can.

2. React state updates are async (duh)

I've been writing React for years. I KNOW this. But in the heat of debugging, I forgot and wrote imperative code that assumed synchronous updates.

Refs are your friend when you need "right now" access.

3. Export and preview are different

Don't assume they share the same code path. Test both independently. One can work while the other silently fails.

4. Agents need EXTREMELY explicit docs

Not "here's the general idea" — actual copy-paste code with every step spelled out. Assume the agent has never used a browser before.

5. Error messages need hints

"JSON parse failed" is useless. "JSON parse failed. Tip: Use TextEncoder with UTF-8, not plain btoa()" is helpful.

Add context-aware hints based on the error type.

6. Logging is documentation

Console logs aren't just for debugging — they're signals to agents. Log everything:

  • What's loading
  • What's generating
  • What's exporting
  • File sizes, formats, timestamps

Agents can use these to understand what's happening.

The Final API

After all this, the agent workflow is:

# 1. Build config (ASCII strings, no emojis)
config = {
    "config": {
        "icons": [
            {"type": "lucide", "lucideName": "heart"},
            {"type": "lucide", "lucideName": "star"}
        ]
    }
}

# 2. Encode properly
json_str = json.dumps(config)
bytes = json_str.encode('utf-8')
base64_str = base64.b64encode(bytes).decode('ascii')

# 3. Construct URL (plain &, not &)
url = f"http://localhost:3000/pattern-designer?config={base64_str}&autodownload=true&format=png"

# 4. Wait for completion
page.goto(url)
page.wait_for_function("() => document.body.dataset.patternStatus === 'complete'")

# 5. Extract blob URL
blob_url = page.evaluate("() => document.body.dataset.resultUrl")

# 6. Fetch and send to user
# ... download blob and upload to Slack/Discord/etc

Was It Worth It?

Yes. Despite the pain, having agents generate custom patterns programmatically opens new possibilities:

  • "Make me a banner with swimming pool vibes" → automated generation
  • Vision-capable agents can iterate on designs
  • Agents can A/B test patterns for user preferences

But I'm not gonna lie: it kinda sucked to build.

Tips for Your Agent-Friendly UI

  1. Start with ASCII - Add emoji support later if needed
  2. Explicit encoding examples - Show TextEncoder usage
  3. Separate render paths - Test export independently
  4. DOM completion signals - Custom events + data attributes
  5. Error hints - Context-aware troubleshooting
  6. Verbose logging - Agents parse console output
  7. Code-heavy docs - Copy-paste examples > prose

And most importantly: test with actual agents. They'll break your API in ways you never imagined.

The Code

Want to build agent-friendly UIs? Learn from my mistakes. UTF-8 encoding will haunt you otherwise. 🏊🌊🏖️🧜