Reliable Data Uploading on iOS
Reliable Data Uploading on iOS4 levels of confidence: simplicity, resilience, continuity, and durability
Reliably uploading data is sort of like brushing your teeth: it’s a dull chore, but you kind of have to do it, or you’re going to get an infected dental nerve. What was I talking about again? Right. Dental uploads. Data uploads. Losing user data through developer negligence around upload hygiene is maybe #2 of the 3 most common Very Bad Things You Can Do™ as a client-side developer. The worst is mucking up auth and logging out all your users. That’ll kill you. Crashes are a common #3. But losing data erodes user trust. Uploading properly is really important to understand. I hacked together a simple open-source companion project that demonstrates all the techniques here. FakeTwitter. 4 tabs, identical UI, but each employing increasingly elaborate upload reliability as we climb the ladder from simplicity, to resilience, to continuity, to durability.
The techniques can be mixed and matched based on your requirements. Usually you’ll have a mix, depending on what data needs to be reliable vs. what’s simple to ship and maintain, because each level introduces growing complexity. I’m sharing practical advice you’ll be able to apply in your day-to-day when working with APIs. I also used the theory directly in my interview grind, specifically in system design loops. The important takeaway I want to give you is to be thoughtful about your networking: consider what level of reliability you actually need for your feature. Once I’m done with you, users will be clawing at your office door, begging you to upload their data. Sponsored LinkMeet Rico by RevenueCat
ContentsLevel 1: Fire-and-ForgetThis is the simplest approach. By far. It’s also the default you’ll begin with for any upload request. Just try POSTing your data using the bog-standard URLSession API. You’ll ship fast, you’ll learn a lot, you’ll mostly get the data where it needs to go, but not always, and that’s fine. Level 1 isn’t just being dumb. Most network requests in most apps, even those shipped by big tech, begin at Level 1. You default to this when loss of data is an acceptable outcome. You should absolutely wrap this f&*ker in a try/catch, log an error when the request fails, and surface some kind of feedback to the user. We’re Level 1 networkers, we aren’t animals. Other than basic hygiene, there’s not too much to say. Just remember that if the network drops, or the server has a temporary issue, your user’s data is just- poof. Gone. Kaput, like my Brewdog shares. Level 2: Retry DisciplineThis is where we start to limber up our system design chops. You’ve likely encountered most of these, but it’s helpful to have the full toolkit laid out formally. We can introduce retry behaviour without seriously affecting our product surface, and without introducing any serious engineering challenges (yet). User-Surfaced RetryIn Level 1, we considered surfacing errors to the user, like an alert. To advance to Level 2, just slap a retry button on this bad boy. This is actually employed by the “true” Twitter app (perhaps you could argue that X is fake Twitter) when your tweet fails to post.
Automated RetryWe don’t immediately need to run crying to the user for help. Automated retries offer a number of strategies. The most basic is a recursive retry with an incrementing counter to attempt a max number of times. Exponential backoff is the most famous flavour of automatic retry, where you double the wait between each attempt, the exponent in this case being t = 2ⁿ, where t is the wait before retrying and n is the number of retries. For the less mathematically-inclined reader, this is doubling every time. If the request keeps failing, we try again after 1 second, 2 seconds, 4 seconds, then 8 seconds, and ultimately fail after 5 attempts. Some errors are a you problem. Perhaps you listened to the Queen of Canada and stopped paying your cellphone bill. Some errors are a me problem. Oh no, the server is on fire, and can’t stop sending 500 responses. We can introduce a circuit breaker in the latter case, blocking requests entirely. This avoids hammering our ailing server and gives it a short cooldown window to stop melting. When your server flails, it can return errors for everyone at once. This can lead to all your users retrying simultaneously, crashing into your server once it’s back up and killing you again. This is called the thundering herd problem. We can guard against this by adding jitter to retries. This is a bit of randomised jiggle added between each backoff to space out retries. This AWS blog post goes deep into jitter strategies (so I don’t have to). If your backend guys are really trying to earn their staff promo, they might supply a Retry-After HTTP response header telling the client exactly when they can retry: Idempotency Keys“Idempotency key” sounds very complicated, primarily because idempotency is a long and scary word. I actually left my ex wife because she said I was idempotent. Consider making a bank transaction. It succeeds on the server, but I miss the 201 success response because the connection drops at the worst possible time. Network error? Automatic retry. I do not want to be sending double alimony, but fortunately idempotency comes to our rescue. Look, I was pretty proud of that joke; if you laughed, please pay me money. Most apps will implement retries and call it a day. Upgrade today for the full production-grade reliability toolkit, which handles flaky networks, backgrounding, interruptions, and when to implement a durable sync engine. Paid members also unlock several nice things: 🌟 Access Elite Hacks, my exclusive advanced content Continue reading this post for free in the Substack app
|
