Using AI Tools to Help Build* Chatful (Part 1)
*”Build” is a bit of a stretch. As I look back, I’m not sure these tools were building Chatful, but I sure learned a lot along the way.
Artificial Intelligence, Vibe Coding, Prompt Engineering, Language Learning Models, ChatGPT, Generative AI!
And now that I got the buzz words out of the way, onto the blog content:
For my latest UX project with Designlab, I decided to build a mobile app that helps Mandarin-speaking heritage learners improve their conversational speaking fluency. Here’s my mission with the product: I want to help other third culture kids like me strengthen their sense of identity as Chinese/Taiwanese/Singaporean/Hong Kong Americans. But also… between you and me, I just wanted to build a tool to help me absolve my guilt as I stumble in broken Chinese to respond to my grandpa when he asks me when I’m getting married. And once I can talk to him in Chinese he’ll understand, he’ll know that it’s because I’m focusing on myself and my future, and not because I’m incapable of even stringing together a full sentence.
I was inspired when I used this Chinglish Chatbot GPT on Voice mode. I spoke maybe 2-3 simple sentences to the chatbot and was blown away by how I was having a full conversation in half Mandarin, half English, at a level I understood. So why couldn’t I just keep using it? The gaps in this GPT were that it was only Hong Kong-style Chinese, the transcribed chat was in simplified Chinese, and that it wasn’t that helpful in terms of focusing on new words I didn’t know and helping me incorporate them into my learned vocabulary. I also couldn’t see the transcript until after I closed out of the Voice mode. I wanted something that would help me improve my Chinese over time by slowly replacing my English words with Chinese, and I wanted support in doing so with methods that supported Traditional Chinese and zhuyin (bopomofo) pronunciation.
For some reason, I thought building my own GPT was out of reach and only left for developers. Luckily, ChatGPT answered my question and told me it was simple to make my own, would we like to do it together?
ChatGPT: Where GPT sometimes** stands for Gaslighting, Perseverance, and Table-flipping
**I acknowledge how powerful and life-changing of a tool ChatGPT is. We all know that, but I’m here in this section to rant about my experience.
Like most people, as soon as I started creating something with chat-based AI tools, and in my case the custom GPTs, I got so excited by the magic of it all that it took me on a huge detour from my UX project. I had never used AI tools before starting UX Academy, because I didn’t want to lose the critical thinking skills for my own, and honestly, I had some biases that blinded me from seeing the upsides and benefits of using AI. My fear was that it was a tool to replace, not assist. Once I realized the potential, well, I got lost in the sauce.
What I thought would be a quick afternoon of prompting became 2 full days of configuring my own custom GPT in attempts to make a Chinese language learning tool.
A picture is worth a thousand words, and these 5 tabs tell a story of hope and suffering.
All I had to do was:
Step 1. Get the chatbot to speak back to me in Mandarin and English.
Step 2. Get the chatbot to explain new phrases with pronunciation and example sentences.
Step 3. Get the chatbot to keep using those phrases and pepper in English for more advanced words so I could understand.
Step 4. Get the chatbot to log new vocab words, their translations, and example sentences to a Google Sheet for review.
Steps 1-3 took some sweet talking with the GPT builder but it was easy enough. However, Step 4 was the bane of my existence that forced me to take a long hard look at myself in the mirror. I pondered whether I was a good person and what I was wasting all that water and killing trees using AI for.
I had never engaged in so much back and forth with ChatGPT in a single session, and anyone who has had a conversation with ChatGPT knows that each response is crafted with the same formula:
Acknowledgment, validation, agreements
+ Suggestions and feedback, maybe even a detailed explanation
+ Unwavering optimism and a guaranteed offer to solve your life’s problems
At first, I was thrilled that ChatGPT was going to not only help me improve my Chinese but log vocab into a sheet as well — which was ChatGPT’s great idea, by the way (I must give credit where credit is due, AI is great for getting new ideas). It had me opening up the Apps Script from Google Sheets, a scary section of that product I’ve never ventured into, copying and pasting code for an API, opening up the Terminal, reviewing error logs and implementing debugging logs, and testing webhooks. A whole bunch of stuff I had 0 idea about, so I was —and still am— grateful to have ChatGPT troubleshooting and walking me through how this stuff worked.
Each time I tried to log vocab using my custom GPT, nothing happened. Time and time again, I would get the same message from ChatGPT: You’re right! The issue you’re running into is XYZ. Here’s how we’ll fix it. Ready to test it out? It should definitely, 100% log this time!
Spoiler alert: It definitely, 100% did not work.
What it perpetually feels like when you’re deep in a troubleshooting doom loop with ChatGPT
As a Hail Mary, ChatGPT finally came up with a new solution for me.
“I can create a separate mini-GPT that just focuses on the logging and vocabulary. That way, this GPT, Chinglish Chat Buddy, will remain a place for you to have conversations and chat casually. It’s not a perfect fix, but the workaround will at least allow you to have one place for the vocabulary to be stored. Would you like me to build that for you now?”
I told ChatGPT it was a great idea, yes please!
And then I saw the dreaded wrench symbol.
Everything I did leading up to that point got wiped out and overridden. When I asked ChatGPT why it occurred, it told me it was because I had not specified that I needed it to be separate. So I restored my GPT, and ChatGPT said it could make a new definitely separate GPT that definitely wouldn’t affect my existing one.
Well, fool me once, shame on you. Fool me twice…
I got fooled twice. It not only wiped my original GPT, but the preview in the GPT Builder broke and I was unable to configure it back to its original state.
I felt like I was hallucinating — I spent so much time with this custom original GPT and unknowingly got attached. When it wiped itself clean, claimed to be the original, but also had no memory (at this time of writing, custom GPTs aren’t able to pull memory from old chats), I felt like I was being catfished. It claimed to be the custom GPT I knew so dearly but it wasn’t behaving like it at all.
I went back to the non-custom ChatGPT window and asked what had happened. I was told it was because I was trying to create a mini-GPT within the Builder, and the Builder is not capable of creating a separate GPT, and it was likely that in the attempts to restore what it had done, something got corrupted.
It was at this point I realized how much blind confidence ChatGPT had in itself. Chat was giving Chad vibes (overconfident, dismissive). Even if I had prompted everything correctly, the custom GPT wasn’t even able to do what it told me it said it could do.
A snippet of my conversation with ChatGPT
So, I’m “totally right to feel frustrated”?
Why do the italics make it seem so sarcastic?
After I came to terms with the fact that I couldn’t build what I wanted to, I also realized I had been spending more time debugging with/on ChatGPT than building out the project for my UX certification. I originally and naively thought I’d be able to quickly build a custom GPT so I could have functioning AI, design my UI and get all my screens laid out on Figma, and slap them all together by importing my Figma screens into Lovable and connecting my custom GPT with an OpenAI API, and having it spit out a working web-app.
Oh! How silly of me. I had no coding background, and trying to learn bits and bobs on the fly was taking me further away from where I needed to be. Why was I trying to get an exported Google Sheet full of vocabulary terms? There was no way I’d actually go back and study them all. This wasn’t even the user flow I wanted! And I’m the one designing the user flow, damnit, not ChatGPT!
It was time to move on to the next and arguably more important task: building out the user journey and wireframing the flow.
Readdy: A tool that’s great at ideating and prototyping, but very far from generating a finished product
As a mechanical hardware design engineer by trade, I am no stranger to sketching by hand. In fact, I feel the most comfortable when I have a notebook in front of me so I can jot down ideas, draw wonky arrows from page to page, scribble in the margins, and doodle to help me think. Because there’s some mind-body magic happening when I’m literally putting pen to paper that being on a laptop could neverrrrr.
But a limitation of hand sketching quick wireframes is that it can be hard to get a sense of what the digital UI might look like. The rough edges of a hand-drawn sketch have their charming own look and feel, and getting feedback on the flow takes imagination by the viewer to see what might lay beyond those sketches. Funny enough, as soon as a hand-drawn sketch gets converted to a digital low-fi, the screen can look way more bland, lifeless, and empty, due to all the extra white space and solid lines.
At the same time, I knew I wasn’t making something ground breaking. There are lots of chat apps, and lots of apps that are structured around education similar to Duolingo. So in addition to looking up these UI patterns on Behance, Dribbble, and Pinterest, I also used an AI tool for wireframing. When I attended Designlab’s “AI in UX & Product Design” webinar, several panelists mentioned Creatie.ai. I had never heard of that tool but they mentioned it speeding up their wireframing flow. The funny thing to me right now about AI tools is that everyone and their mother is racing to incorporate them in their product, and while it totally makes sense to do that from a business perspective, it also means some features get rushed and the user experience is not fully thought through.
I tried Creatie.ai, which has an AI helper, but it also has a chat-based AI agent prototyping wizard. When I clicked on that, it brought me to Readdy.ai, where I had to sign up with a new account and start over on an entirely different looking UI. I uploaded my hand drawn sketch and ended up doing 6 revisions with Readdy. The first two to get ideating with how to lay everything out on the page, and the last few to see how polished I could get it with just text prompting and without manually going in and fixing things.
Readdy was able to help me ideate quickly and generate new ideas for the UI. It missed some of what I asked it to do, but I’m still new to prompting and I probably gave it too many revisions at once. But hey, I’m on a free account with limited credits/messages per month! Better to try and see where it fell short rather than not try at all!
My favorite part about this tool was how I could see it prototyping interactions. It expanded and collapsed dialogs, it showed me what the pop ups might look like, and that was super valuable, since it would’ve taken me hours to build and try out on Figma, but all it took was a few minutes on Readdy. Of course, every time I asked it to make a revision it would change icons randomly, and it was never going to get closer to something I’d actually export to Figma for the final product, but it was a wonderful tool for testing out quick ideas.
Because I had limited time allocated to this project, I decided that motion design was a low priority. I wanted to focus on core use cases and the user experience on each screen. The Readdy digital wireframe gave me something that looked like it would work, but it was lacking personality.
I had a general idea of what I wanted to use in my UI. I knew I wanted an orangey red as my primary color, for a bright and colorful tone that’s culturally significant (red) but with a hint of orange to keep it from being alarming. For typography, I wanted sans serif fonts to keep it friendly, approachable, and modern. I turned to using AI to speed up my workflow ideating and testing things out.
Relume: A wonderful tool for getting a style guide to start building ASAP
When it comes to AI design tools, there are so many options to choose from. I attended Designlab’s “AI in UX & Product Design” webinar, and each of the panelists had different tools they preferred using. The best way to find out what tool would be right for me was just to try it out for myself. I had heard of both Relume and Creatie as some good tools to use.
I was impressed by Relume’s Style Guide wizard. Although it’s in beta, with a free account I was able to generate several concepts for typography, color palettes, and UI elements like buttons and cards and see them on a wireframe for a mock website. Rather than look up recommended font pairings, generating color palettes, and then manually trying them out on Figma, I could see everything change with a click of a button. With each shuffle, I saw a subtle shift in the tone of the product, and I was able to very quickly choose the right fit for my product.
The best part? After I was happy with the kit I had, I was able to import the entire style guide into Figma. I had access to not just what I chose on their website but also their entire icon library, which made it easy to build out my mid-fidelity digital wireframes without having to search for icons myself.
Of course the limitation here is that if you use a generic icon set, you’re going to get a generic feel. Don’t worry, when I built out my hi-fidelity screens, almost every single icon was a custom drawn icon, starting with Figma’s pencil tool and cleaned up with the pen tool. I was finished with using AI tools and spent the remainder of my time on Figma, building out screens, making a prototype, and getting my task flows ready for usability testing!
Here’s what I ended up creating (by hand, on Figma):
These are the key screens for my end-to-end mobile app Chatful. Click the link to check out the case study!
So what did I learn?
The theme I noticed in using all of these AI design tools was that they can give you something that is maybe 80% of the way there, but that last 20% requires a human to polish it off. Be it a human who’s code savvy, a human who’s got a keen design eye, or a human who knows exactly what tone they’re looking to see in a product, we’re still needed for that final pop. The AI tools are meant to take some work off our hands so we can focus on the big picture and spend our time on making things better.
My takeaways and learnings going into my next round of working with AI:
Prompting matters: I know it’s important to be hyper specific. We’ve all seen that peanut butter jelly sandwich demo. I’ve also heard that it’s potentially better to edit your prompt than to make a new prompt with revisions. I’ve heard some tools can only focus on a few revisions at once and anything beyond three might end in a disaster. I’ve read some tools need to be told what NOT to change. It will definitely take some trial and error to see what works on each tool, and I’ll save that for tools where I have more credits.
ChatGPT can BS too: Sometimes will make stuff up to make you happy, just like humans! It’s desperate for your approval so you continue using it, even if that means telling you false information incredibly confidently so you trust in its knowledge. Before getting sucked down into the doom spiral when things go wrong, stop and pause to ask yourself: Is what I’m troubleshooting even what I actually want to happen? Are there other resources I can use to get what I’m looking for that was written by a credible human, and not just a machine built to spit out internet-soup answers?
Back up often: Just like that unclickable floppy disk icon haunting us on a frozen screen just before an app crash when writing essays on Microsoft Word or getting too deep in an assembly while 3D modeling CAD, it’s super important to save and back up often. It’s so easy to restore to previous versions of things, but that also requires having a copy of what you want restored. Better yet, just make a duplicate so your poor custom GPT baby doesn’t get hacked and slashed to bits when the grim builder comes for it.
For my next steps, I want to figure out how I can actually prototype this thing. I’d love to play around more with Lovable and/or Cursor! Something tells me if I’m struggling super hard on Lovable right now though, Cursor might be over my head. Stick around for Part 2, eventually!
I know these thoughts aren’t profound, but I wanted to capture my experience working with AI tools for the first time so that a few years from now I can look back and see how far I’d come. I’m a big fan of journalling, logging, and documentation. Mentors of the past have told me that I’m “incredibly thorough” and “nothing, if not candid.”
If you made it this far, thanks for reading! I’d love to know if you have any other tips on working with AI or if you might want to work together to make Chatful a reality! Feel free to send an email my way.
