351: From Overload to Opportunity
Download MP3Hey. It's Arvid, and this is the Bootstrap Founder. This episode is sponsored by pedal.com. More on that later. As founders, we often find ourselves in situations that test our foresight and adaptability.
Arvid:And this week, I experienced one such moment that served both as kind of a wake up call and a valuable learning opportunity for me. I think it's a story that highlights the importance of anticipating potential issues in your software business, even when they seem unlikely, and that's the key point. Right? These little random things, they're probably not gonna happen. Well, they will happen.
Arvid:So grab a cup of coffee and let me share this cautionary tale with you today. It all started with a user of PodScan, my podcast monitoring service, and PodScan sends out email notifications whenever certain words, keywords, or phrases are mentioned in the podcast. So for most users, this means a handful of emails a day, maybe a dozen if they are tracking popular terms in their niche. But this particular user, well, they decided to set up alerts for words like Google and India and America. I don't know why, but, you know, they did.
Arvid:Now if you're thinking, wow, that is going to generate a lot of emails, you are absolutely right. And that's exactly what happened. Within a couple days, this single user was receiving thousands of emails daily. Sometimes, like, I think 5,000 emails a day was a lot. To put this into perspective, most users really get a dozen notifications a day.
Arvid:And most of those who do have set it, so that they get get summary emails, instead of getting individual notifications. But this user was getting more than an hour than others would get in a month, And it did not change that particular setting. And I didn't change it for them because, yeah, we'll get to that. At first, I didn't realize what was happening. It wasn't until my email provider actually slammed on the brakes that I understood the magnitude of this whole situation.
Arvid:They cut off my email sending capabilities entirely, and they cited concerns about server deliverability issues. And suddenly, I couldn't send any emails at all. Not even critical ones like login verifications or password reset emails. And my initial reaction, well, I was frustrated. I thought, hey, I'm paying you to send these emails.
Arvid:Why are you cutting me off? And I think I tweeted about it too. I didn't say who it was, but I did. And as I dug deeper, the reality of the situation became clear. This wasn't just a technical issue or some business choice from the email provider that I was using.
Arvid:It was a potential threat to the reputation of my entire domain, and it was my fault. The consequences of this incident were far reaching from the start. My domain was teetering on the edge of being blacklisted as a spam source because I think the person who got these emails was on Google Mail like most people are. And Google starts to really look into these things when they happen. Right?
Arvid:They immediately notify the network and all of that. And it could have been very, very easy for me to get on the the blacklist. Not necessary there are multiple different lists, like allow lists, block lists, and there's a yellow list or something was, like, in between. And I made it on that one at least for the day. And had I progressed even further sending more emails, it would have been a real problem.
Arvid:If that had happened, it could have been catastrophic for PodScan. Like imagine building a business that relies on email communication because it's about real time communication, only to find yourself unable to reach your users. It's a nightmare, and it's a nightmare scenario for any founder. If you really wanna reach your users and you can't. But here's the kicker, it really was not a malicious attack or anything.
Arvid:It wasn't even intentional misuse by that particular user. It was simply a user who didn't fully grasp the implications of their actions because I didn't communicate them, and it was combined with my oversight in not setting appropriate limits. And this experience forced me to confront an uncomfortable truth. As founders, we are responsible for anticipating and preventing these kind of issues, even when they seem unlikely. It's not enough to build a product that works under ideal conditions.
Arvid:We need to build systems that are resilient to unexpected use cases and potential misuse, intentional or not. And realizing the gravity of this situation, I immediately jumped into action. The first step was damage control. I immediately stopped sending emails for this particular user to prevent further issues with my email provider, and then I switched email providers to ensure that critical business functions, like user sign ups would still continue uninterrupted. And the real work actually came next.
Arvid:I had to implement safeguards to prevent this from happening again. And here's what I did with this particular thing. You might find this interesting just as a consequence of this. I implemented daily email caps. I introduced a daily limit to the number of emails that can be sent to any given account.
Arvid:And by account on PodScan, I mean a team. Could be any number of people, but once this limit is reached for the team, email sending stops for that particular team, all users on that team for the day. I have a daily cap and I have an hourly cap. So the idea is the daily cap is kinda 3 times as high as the hourly cap. So if there is a lot of talk about a particular keyword in any given hour, they still get those emails.
Arvid:And then if it continues for a couple hours and more podcasts are released, they get more emails from that, but they reach the cap than sending stops. And I also added a clear notification in the UI, because people still very much click the link to PodScan in their emails to see more information. So I added a little bar in the UI that informs them that they have reached that daily limit and tells them that they can either do something about that, either ask me to change the limit or change the settings so that they get a daily summary email instead of all these individual emails. And for users who genuinely need higher limits, I introduced options like these summaries. Right?
Arvid:They they exist for that particular reason, and I highlighted them more in the UI. And for enterprise customers or those on higher tier plans, I just made it possible to set account specific exceptions to these limits. So there's a baseline limit of, I think, 10 an hour and 25 a day in terms of emails that I send with PodScan. And if somebody really wants more of these emails, well, they can ask me and I can give it to them, or they can just use the webhook feature that they have anyway where I send the full information of an mention of an alert as a webhook to a certain URL, and then they can dispatch it into, I don't know, a Google Sheet or an email if they want to. But I don't have to send these emails manually or automatically at this point anymore.
Arvid:They can do whatever they want with it. So I give people a lot of options, but they're all based on limiting what I do. Because the thing is, I have no problem sending 50 emails to a a particular kind of customer if they want that. But at this point, I have hundreds of customers. Am I gonna send 50 emails a 100 times?
Arvid:Now that all of a sudden that's 5,000 emails. And first off, that costs me something, like emails aren't free. And if I send 5,000 emails and they're all going 80% of them to Google Mail, like, some some kind of Gmail system anyway, they're gonna think, this guy is sending a lot of emails. Right? And I I was talking to a couple people on Twitter about this.
Arvid:They were saying, Google really tracks engagement with emails to genuinely figure out if it's a scam or if it's just spam emails or something. And engagement is the most important part. And for a notification email, which is transactional. Right? It's not a marketing email that people would ignore.
Arvid:It's supposed to notify somebody of something that is going on. For that, I would really, really need a lot of people clicking on it. The idea is for an email to be considered engaged, people have to click on it. Click on some link, open it, open rate. Right?
Arvid:That's the idea. They have to open it, click on the email initially, and then click on something in the email ideally. And I think the open rate of a summary email is much higher than of 10 individual emails. But, you know, if somebody really wants to look into every single email, they can do this too. Anyhow, this incident got me thinking about the broader implications for software businesses in general.
Arvid:It's not just about email notifications, right? It's about any action in your system that could potentially be overused or misused. And as founders, we need to approach our products with this mindset of defensive design. This means thinking through potential edge cases all the time and implementing very reasonable limits on all user actions. That's the key.
Arvid:Every single thing that people do, you kinda have to think about, well, when is it too much, Right? When does it, as consequence, cause damage, cause bloating, cause financial problems? When is something that somebody does or that somebody asks my software to do a potential problem. And here are some areas where I think this applies most obviously. I think API requests is the best example for this.
Arvid:Right? You wanna limit the number of requests that a user can make in any given time frame. Because everybody wants to scrape your business. Everybody wants all your data, all the data they can get. And sometimes, just a normal API request when repeated like a 10 or 20 or a 1000 times a second can really do massive resource damage to your business.
Arvid:People might just they might built, an API client for their business that all of a sudden does way more than they wanted it to do because, you know, people are developers, they make mistakes. And you need to preempt this by saying, you get 60 requests a minute. That's 1 a second, that means if you really wanna go super hard for, I don't know, 4 seconds sending 15 requests each, fine with me. But after that, you're gonna get a 4 29 HTTP, response. Just to say, hey, here's how much you have per minute.
Arvid:Here's how much you used. Deal with this. API requests need limit. Rate limiting. That's the answer.
Arvid:Right? Rate limiting generally is a good answer. And rate limiting does not just happen on APIs by the way. I just want you to know that if you build a software business where people go to certain routes, like they go to their dashboard, they go to their billing page, they go to their profile, they go to the project that you have, they go to some part of your website. Any of these routes should have a rate limit.
Arvid:That is very very important. First off, if you have any kind of directory of data, people would like to scrape it. Like PodScan in particular lists all the podcasts and all the episodes. Like to somebody else, this might be very interesting. Right?
Arvid:It's very interesting to scrape and do whatever they want with it. I mean, they could just get it from the API if they wanted to, and they could just ask me and I could give them an an export, sell them like a full on SQL file or something with all the data in it. But, you know, people are interested in building their own solutions. So any part of your website could potentially be scraped by an automated system. And to prevent this from happening, I would highly recommend setting reasonable limits on all page views on your website.
Arvid:That includes your homepage, your landing page, if you have control over this. Like on a per IP level or on a per I don't know where where you would get it, like, per login cookie credential level, always set rate limits. I think I have, like, 12 requests per minute on most of my, websites that I have, like, in the dashboard. People are likely to click, click, click, click, click, and then they are where they need to go. And maybe it's it's 20 or maybe it's 30 clicks, but they will see, okay, I clicked too much.
Arvid:Let me just wait for a second. This is still better than having somebody scrape my entire database and siphoning off massive amounts of value. That's defensive design. And data storage is also a thing. Right?
Arvid:Set caps on how much data a user can store and how many items they can create. To me, this is important because it's part of my tiering system for my pricing. But, generally, it would be good to have at least a notion of when somebody is completely using way more data than you thought they would. Right? If you assume that a user in your in your business, of your particular SaaS business has, like, up to 5 projects or has a project with a couple items in it, then if you see somebody who has 50,000 projects, something is going on.
Arvid:Right? There should be a limit there, soft or hard. A soft limit is when you reach it, you get a notification. A hard limit is when you reach it, you can't do more, but have a limit in place. And, generally, anything that causes a lot of resource intense operations, like actions that people take, transcribing an audio file or uploading a video, stuff like that, have a limit on that as well.
Arvid:How many times can it happen? How many times per day? How many times per month for that matter? How many times per credit that people have on their system? You you always should have a limit.
Arvid:Any action should have a limit. But it's not just about setting limits, it's about creating a system that then gracefully handles these limits when they're reached. And I think this includes very clear communication to users when they are approaching that limit or have reached it, And you should have that very clear in the interface of the the software that you create. And if it's an API, you should have these these headers, like these kind of, headers in the response that indicate where the limit is and how close they are to reaching it, so they can use automated systems to figure out if they need to kind of throttle their requests a little bit. And you should always have alternatives or upgrade paths for users who legitimately need higher limits.
Arvid:Particularly with PodScan, I'm looking for bigger and bigger customers. I'm trying to get into the newsrooms of the world and get into the organizations that deal with a lot of data. They have other requirements. They may be fine with getting hundreds of emails. They may actually need or want these emails.
Arvid:So I need to be able to increase my limits. And when you're done with all of this, you need to implement alerts for the team, for yourself. When limits are consistently being reached, not just once, but every single day or multiple times an hour, because this indicates a need to adjust your whole system, your pricing tiers, or the limits themselves. And one of the trickiest aspects in implementing these safeguards is maintaining a positive user experience in all of this, because you don't want your limits to feel restrictive to the point where they frustrate users or hinder their workflow. And this is where the art of product design comes into play, I believe.
Arvid:I think it's really about how you communicate it. The goal is to make these limits feel natural and reasonable. For example, if instead of just this hard stop when a user reaches a limit, you might implement a kind of cool down period or offer a click here to extend your limit, kind of one time extension. Give people a feeling of agency in that moment, but also make it clear that there is a limit and there's a limit for a good reason. I communicate this with PodScan when I just say, hey.
Arvid:I don't wanna spam your inbox, which is why we stopped sending your emails. Right? And if people then want to say, no. It's fine. You can spam my inbox, please.
Arvid:They can reach out to us. I have a link there, like, contact us for a change or something. So people have agency in that moment. That's very important. And it's crucial to be transparent about the limits.
Arvid:Include them in in your documentation, in your onboarding process, in your interface itself. And perhaps even in your marketing materials where it could be both a thing where people say, okay, they are thinking about the viability of their email sending system, they are thinking about my inbox and not to spam me, that's really nice, I'm gonna look into this product. I think this sets clear expectations and I think it can be positioned as a feature that ensures fair usage and system reliability for all users. And clear expectations are an important thing. They are also very critical for your payment provider, which is why I wanna give a quick shout out to paddle.com, the sponsor for this episode.
Arvid:I used to cobble together my own solutions in the past when it came to payment. I used like APIs from all kinds of services, building my own UI or buying some UI out there and kinda putting it all together. Because I thought I knew what a good billing portal looks like, but I had no idea what people really needed and that's, I guess, the founder's curse. We think we know until the reality of the market hits us right in the face with real user behavior. So if you would like to be hit in the face less and instead enjoy working on your actual business, look into Paddle as your payment solution.
Arvid:I run it on several of my products because it's easy to integrate and it takes care of a whole lot of things other payment providers don't. Paddle handles taxes, invoicing, and their bidding portal is a beautiful and highly converting piece of technology that you can just plug into your SaaS and use right away. So check out pedal.com. It's gonna give you peace of mind and drive you revenue, and that's what a founder wants. As I worked through my challenge over the last few days with this email thing and, you know, everything beyond, I couldn't help but think of other examples in the tech world where similar issues have arisen.
Arvid:I think as I am very active on Twitter and active in the Twitter developer community, I've been following the whole situation, let's call it that, with access to the Twitter API as a example here. Twitter in the past, and I mean, like, pre Musk, has constantly evolved their access tiers, the limits and all of that to balance providing value to developers and protecting the system from abuse. It was always a struggle and they always had a lot of bots, they always had a lot of people who just played around with it and did weird stuff, so you could always see these limits changing. And then, and this is now the Elon Musk era of of Twitter ownership, they comically exploded most businesses that were built on the API by making it cost $42,000 a month. That was weird.
Arvid:And it was kind of painful to see. The developer community did not like that at all. But I bet there was a reason to make access harder. Maybe not $42,000 worth of harder, but I've seen fewer scammy build your own audience automatically tools around. I think that's a good thing.
Arvid:So in a way, as much as it was painful for a lot of entrepreneurs to not be able to build tools on top of an API that was easily accessible anymore, these limits, and I don't mean like rate limits here, I mean like financial access limits, did impact the quality of tools built on top of the Twitter API. Out of the tools that can actually afford it that were making enough revenue to be able to pay this horrendous fee, the ones that are still sticking around, well, they really need to make it work because they need to pay a lot of money to Twitter or to x at this point. It's very interesting. Right? Like, you set limits, and it changes the way that people use your product.
Arvid:And then you play with the limits, and you have an impact on who gets to use your product and how they use it. I think that's really cool. And this is just Twitter. Right? Like, think about other services like Dropbox out there, how they handle file upload limits when you reach that limit.
Arvid:They just don't cut you off either at that point. I recently got to this point. I think I have a terabyte or more in my Dropbox. They offer a clear upgrade path from early on when you're, like, 80% in when you're approaching this limit. They they make it very clear that there is a way out, and here's how to pay it.
Arvid:I think just those two examples, they made me think about that I'm not alone in facing these challenges. Maybe at a wildly different level, but as a community of founders and builders, we can learn from each other's experiences and best practices. And I think this whole story that I'm telling you here about this little problem that I had is one of them. I hope you find some value in understanding, oh, limits. To think about limits.
Arvid:Yes. And with this incident being stressful and potentially dangerous for my business, it still ultimately led to a lot of positive outcomes, which is also important. I wanna stress that. Right? There's a challenge, you overcome it, and then you better yourself.
Arvid:And the business gets better as well. I think PodScan is more resilient. I implemented these safeguards, and PodScan now is more robust and better equipped to handle unexpected usage patterns. Because I have limits in place and I have alerts in place. That's the important part.
Arvid:I also implemented a new notification system for limits. And this has opened up more channels of communications with my users. This little notification bar that I built, well I built it in a way that it's flexible. That I can display more direct notifications that I wanna give to people and this will lead and already has led to better engagement and feedback from people because it's right there, right on top of the screen. I just had no use for it yet up until this point, so I hadn't built it.
Arvid:Now I have it, and now I can use it. Right? It's a benefit for the business altogether. And by segmenting my usage limits across these different plan tiers, I've also created a clearer upgrade path for users who need more capacity. If people are on my essential plan, they want more, they want more emails, they want more whatever, I can tell them, hey, your first upgrade is free.
Arvid:Because right now I just implemented it. Right? They they had no choice in this when they signed up for the service. But next time you ask me for this, I would like you to upgrade to the bigger plan. So there's a financial benefit for the business.
Arvid:And also more commitment from these people because they know, okay, I'm using it more and more. I'm getting more value out of it. This value is clearly proposed, so I'm gonna pay more money for it. I've seen this happen with other features in PodScan several times already, so I'm gonna do it with these limits as well. And then finally, I guess, as a founder, this experience has made me more proactive in anticipating potential issues.
Arvid:Not just in PodScan, but in all my future endeavors. Right? Whatever I do, I'm gonna think about what actions can be taken or what automated actions can be instigated by my users that will cause potential problems for me down the road. Just thinking in edge cases or in consequences is really, really helpful. So as I wrap up the story, I guess, I wanna leave you with a few takeaways.
Arvid:First one is always anticipate the unexpected. That's what entrepreneurship is. Always be thinking about how your system could be used or misused, intentionally or unintentionally, by negligence really, in ways you didn't initially envision. Like, I did not think that somebody would put up, an alert for the word Google. Who does this?
Arvid:I mean, this user then. And they probably just wanted to check it out, wanted to see, like, how many results there would be or whatever. I haven't talked to them about it. I've just, like, made couple changes to make it more feasible. But, like, there are so many podcasts every given day.
Arvid:There's 50,000 podcasts that come in. Like, there's thousands of them where people just say, hey. I googled this or then I went on Google. So it will be triggered. You have to think about data avalanches.
Arvid:And, yeah, should have thought about it earlier. And I think that's that's another point, that you have to implement these safeguards very early. Don't wait for a crisis to put limits or protections in place. I think it's much easier to relax limits than to impose them after an issue arises. And also, like, particularly if people pay for your business, it always kinda stings a little bit if a new limit is added on top of what you thought was unlimited.
Arvid:You might wanna set these limits from the start to just be clear. And you have to be transparent with your users about limits and restrictions. Clear communication can turn a potential point of frustration like this into an opportunity for engagement. Hey. I'm I reached my 25 emails a day.
Arvid:Very cool. Could I maybe refine my keywords a little bit so I get fewer notifications or could we turn this into a summary? Like, all of these are communication opportunities that I have with my customers now when they see this little bar, little banner. And they see, okay, I'm getting too many emails. What can we do?
Arvid:And just stay flexible. Be prepared to adjust your limits and your policies as your product and user base evolves. What works today might not be sufficient tomorrow. And I hinted at this earlier. Right?
Arvid:If I go into bigger customers, if I go into more enterprise, then I have maybe fewer customers, but they are bigger. So I can relax my limits so they overall kind of I sent the same amount of emails, but it's just on more per customer of a basis. The idea is if you go into different markets with different expectations, your initial assumptions need to also change and the consequences of these assumptions, the limits that you set need to change as well. Every obstacle you face in this particular path is an opportunity to make your product and your skills as a founder stronger, so lean into this. Always consider, this didn't work or this is not going the way I thought.
Arvid:Why is that and how can I make this more useful to me? Remember, building a successful software business is this continuous learning journey and improvement. You probably have noticed this. Like I've been talking about my challenges on this podcast for months and every week there's something new. There's always something.
Arvid:So and I embrace it. I embrace these challenges. I learn from them and I use them to build something even better every single time. As for me, I'm grateful for this experience. It's reminded me of the importance of just staying vigilant and proactive in product development.
Arvid:Think about what other people will do. And who knows, maybe this story will help you avoid a similar pitfall in the future. Keep building, keep learning, and remember, every challenge is an opportunity in disguise. Just think about what people will do and you will know what to do yourself. And that's it for today.
Arvid:Thank you for listening to the Bootstrap Founder. You can find me on Twitter at avidkahl, a r v I d k a h l. You'll find my books on my Twitter course there too. And if you wanna support me on this show, please tell everyone you know about about podscan.fm. And leave a rating and a review by going to ratethispodcast.com/founder.
Arvid:Makes a massive difference if you show up there because then the podcast will show up in other people's feeds. Any of this will help the show. Thank you so much for listening. Have a wonderful day. And bye bye.
Creators and Guests
