Desktop-Lite MCP Playwright Development Environment

Inspiration: This project adapts the Playwright MCP + LLM browser automation workflow (see Simon Willison’s blog post, inspired by Armin Ronacher) — but here, it’s set up for GitHub Copilot in an isolated devcontainer.

A fully-featured development container that combines visual desktop access with AI-powered browser automation using the Model Context Protocol (MCP) and Playwright.

🚀 Features

• Visual Desktop Environment: Complete desktop environment accessible via web browser or VNC client
• Pre-installed Google Chrome: Ready-to-use browser with persistent profile
• MCP Playwright Integration: AI-powered browser automation using the Model Context Protocol
• Shared Browser Session: Developers can interact with the same browser session that AI agents use
• Persistent Configuration: Chrome profile and settings are maintained across container restarts

🏗️ Architecture

This development environment is built on:

• Base Image: mcr.microsoft.com/devcontainers/typescript-node:1-22-bookworm
• Desktop Environment: Fluxbox window manager with noVNC web access or VNC access
• Browser: Google Chrome with persistent user data directory
• AI Integration: MCP Playwright server for browser automation

📋 Prerequisites

• Docker
• VS Code with Dev Containers extension
• GitHub Copilot (recommended for AI assistance)

🛠️ Quick Start

1. Clone the repository:

   git clone https://github.com/capi/devcontainer-desktop-lite-mcp-playwright.git
   cd devcontainer-desktop-lite-mcp-playwright

2. Open in VS Code:

   code .

3. Open in Dev Container:

   • Press Ctrl+Shift+P (or Cmd+Shift+P on Mac)
   • Type "Dev Containers: Reopen in Container"
   • Wait for the container to build and start

4. Access the Desktop Environment:

   • Web Access: Open http://localhost:6080 in your browser
   • VNC Client: Connect to localhost:5901 (password: vscode)

🌐 Desktop Access

Web-based Access (Recommended)

• Navigate to http://localhost:6080 in your host browser
• Default password: vscode
• Full desktop environment runs in your browser

VNC Client Access

• Use any VNC client to connect to localhost:5901
• Password: vscode
• Provides native desktop experience

🤖 MCP Playwright Setup

The environment includes a pre-configured MCP Playwright server that enables AI-powered browser automation.

Configuration

The MCP server is configured in .vscode/mcp.json:

{
    "servers": {
        "playwright": {
            "type": "stdio",
            "command": "npx",
            "args": [
                "@playwright/mcp@latest", 
                "--browser", "chrome",
                "--user-data-dir", "/workspace/devcontainer-desktop-lite-mcp-playwright/.chrome-profile/"
            ]
        }
    }
}

Browser Profile

• Profile Location: /workspace/devcontainer-desktop-lite-mcp-playwright/.chrome-profile/
• Persistence: Profile data is maintained across container restarts
• Shared Access: Both AI agents and manual users share the same profile

💻 Manual Browser Usage

To manually start Chrome in the desktop environment:

1. Open a terminal in the desktop environment (Fluxbox)
2. Run the following command:

   google-chrome --user-data-dir=/workspace/devcontainer-desktop-lite-mcp-playwright/.chrome-profile/

This ensures you're using the same profile that AI agents interact with.

🔧 Development Workflow

AI-Assisted Development

1. Use GitHub Copilot with the provided instructions in .github/copilot-instructions.md
2. AI agents can perform web searches, navigate websites, and interact with web applications
3. Developers can observe and assist AI actions through the shared desktop environment

Manual Interaction

1. Access the desktop environment via web or VNC
2. Interact with the same browser session that AI agents use
3. Provide manual input when needed (logins, captchas, etc.)

📁 Project Structure

├── .devcontainer/
│   ├── devcontainer.json    # Dev container configuration
│   └── Dockerfile          # Container image definition
├── .vscode/
│   └── mcp.json            # MCP server configuration
├── .github/
│   └── copilot-instructions.md  # AI assistant guidelines
├── .chrome-profile/        # Persistent Chrome profile (gitignored)
├── .gitignore             # Git ignore rules
└── README.md              # This file

🔒 Security Considerations

• Chrome profile data is excluded from version control (.gitignore)
• AI agents are instructed to avoid submitting sensitive project information to external websites
• Cookie consent popups are handled by accepting only necessary cookies
• The browser session is shared between AI and human users for transparency

🐛 Troubleshooting

Desktop Environment Issues

• Can't access desktop: Check that ports 6080 and 5901 are properly forwarded
• Display issues: Try refreshing the web browser or reconnecting the VNC client

Browser Issues

• Chrome won't start: Ensure the profile directory has proper permissions
• MCP server errors: Verify that the MCP configuration is correctly set up in VS Code

Container Issues

• Build failures: Ensure Docker is running and you have sufficient disk space
• Permission errors: Try rebuilding the container with Dev Containers: Rebuild Container

🤝 Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Test in the development environment
5. Submit a pull request

📄 License

This project is licensed under the MIT License.

🙏 Acknowledgments

• Dev Containers for the development container framework
• desktop-lite feature for the desktop environment
• Playwright MCP for browser automation capabilities
• Fluxbox for the lightweight window manager

---

Ready to start developing with AI-powered browser automation in a visual environment! 🎉

---

This README was crafted with the help of GitHub Copilot AI — fitting for a playground dedicated to collaborative, AI-driven development! 🤖✨