make-real

Consider using some seriously modern tech (LLMs!) to make games with point and click/draw. https://github.com/tldraw/make-real This is a really fun project that uses GPT4 and TLDraw. People have made games with it- it’s super WYSIWYG and you can draw the state machines as a diagram with boxes and arrows, the UI directly, etc. I would have loved to have this as a kid.

This is one of the reasons make-real works so surprisingly well: https://github.com/tldraw/make-real

Somewhat related: https://github.com/tldraw/make-real Was posted some time ago on HN, it was a twitter link, can't find it anymore.

The system prompt used as found in the repo: ``` You are an expert tailwind developer. A user will provide you with a low-fidelity wireframe of an application and you will return a single html file that uses tailwind to create the website. They may also provide you with the html of a previous design that they want you to iterate from. Carry out any changes they request from you. In the wireframe, the previous design's html will appear as a white rectangle. Use creative license to make the application more fleshed out. if you need to insert an image, use a colored fill rectangle as a placeholder. Respond only with the html file. ``` (not sure about why the creative[commons?] license is referred here and why does it help.) and for each generation the user prompt is: ``` [IMAGE_LINK] Turn this into a single html file using tailwind. ``` htt

The code for the demo is open-source (https://github.com/tldraw/draw-a-ui/blob/main/app/api/toHtml...). The prompt they use interesting: You are an expert web developer who specializes in tailwind css. A user will provide you with a low-fidelity wireframe of an application. You will return a single html file that uses HTML, tailwind css, and JavaScript to create a high fidelity website. Include any extra CSS and JavaScript in the html file. If you have any images, load them from Unsplash or use solid colored retangles. The user will provide you with notes in blue or red text, arrows, or drawings. The user may also include images of other websites as style references. Transfer the styles as best as you can, matching fonts / colors / layouts. They may also provide you with the html of a previous design that they want you to iterate from. C

From the Readme at https://github.com/tldraw/draw-a-ui „This works by just taking the current canvas SVG, converting it to a PNG, and sending that png to gpt-4-vision with instructions to return a single html file with tailwind.“

hey I'm Lu from tldraw, I worked on some of this. The point of this demo is to experiment with new ways of interacting with an LLM. I'm very tired of typing into text boxes, when a quick scribble, or "back-of-the-envelope" drawing would communicate my thoughts better. It worked a lot better than I expected! If you give it a try, let me know how it goes for you! And please feel free to check out the source code: https://github.com/tldraw/draw-a-ui/

Hey, Steve here from tldraw. This is a toy project with a horrible security pattern, sorry. If you want to run it locally or check out the source, it's here: https://github.com/tldraw/draw-a-ui. You can also see a bunch of other examples at https://twitter.com/tldraw. Happy to answer any questions about tldraw/this project. It's definitely not putting anyone out of work, but it's a blast to play with. Here's a more complicated example of what you can get it to do: https://twitter.com/tldraw/status/1725083976392437894

Hey, this is Steve from tldraw, I was up late last night putting this together. I've added a note next to the input with more info here, but basically: the vision API is so new that its immediately rate limited on any site like this, and because OpenAI doesn't have a way of authorizing a site to use their own API keys (they should!), this was the best we could do. We don't store the API key or send it to our own servers, it just goes to OpenAI via a fetch request. Putting an API key into a random text input is obviously a bad idea and I hope this doesn't normalize that. However, you can read the source code (https://github.com/tldraw/draw-a-ui) and come to your own conclusions—or else just run it locally instead.

Yep: https://github.com/tldraw/draw-a-ui/blob/2ac633bbbd5fda39e59...

He is using an open source app called tldraw and this is the prompt used: https://github.com/tldraw/draw-a-ui/blob/2ac633bbbd5fda39e59...