Mo Users, Mo Problems
Over the past several weeks, I’ve been getting progressively more frequent emails from the Binary Canary monitoring service letting me know that CalTopo was down. It was sometimes unresponsive for extended periods of time, but since it generally came back on its own, I did what any good developer would do and ignored the problem until it reached red alert levels.
After spending 2 days a very long ways away from cell reception for SAR, I came back to a flurry of emails, including another Binary Canary outage warning – and CalTopo still appeared to be down, 5 hours later. This was the tipping point; something had to be done.
Throwing together some changes and deploying them with a half-assed amount of testing was something, so that’s what I did. While it solved the performance issues, it broke imports and exports – something which CalTopo users let me know in short order. Now that the fire drills are over, here’s a quick recap of the issues.
One problem was in the way I sent line/polygon points from the server to the browser. Instead of using a numerical array, each line point was a separate object. This isn’t a big deal in and of itself, but the JSON library I’m using to serialize everything created a heavyweight map object for each lightweight waypoint object, resulting in a huge amount of object churning relative to the memory available. Points are now sent as an array, which doesn’t have that overhead.
The second problem was – and possibly still is – with supplying cursor elevation. Because I push elevation data to the client as a plain-text array of numbers, the 256px tiles are significantly larger than PNG or JPG map tiles. As an added bonus, because elevation data went through the same server-side process as shapes and markers, the browser was instructed not to cache any of it. Every page load generated a flurry of elevation requests.
 |
| One small checkbox, one giant bill. |
Since turning the “Include Elevation” checkbox on by default, it turns out I was spending $150/month just to send elevation data to clients. Oops! I have a rather hefty Amazon bill, most of which is reimbursed by smartphone apps that license my maps. Amazon makes it difficult to separate EC2 data charges (my server) from S3 data charges (map tiles), which is why a number several times larger than my monthly beer budget had slipped by unnoticed.
Now that elevation data is getting cached, I’ll give it a week or two and decide whether I need to go back to having the checkbox disabled by default; the answer will probably be yes.
What this incident has made clear is that I need to find a way to make CalTopo commercially viable. I’ve asked people to send donations to BAMRU because I thought that as a 501(c)3, it would encourage more liberal giving. However, even if the donations BAMRU receives were funneled directly into operating the site, they would still be insufficient to cover my monthly operating costs. Fortunately there are commercial map licensees for that, but while they pay me enough to cover my fixed costs, they’re not really not a profit center.
When asked about my plans for commercialization, I’ve always maintained that while I don’t want to cover the site with ads or turn it into useless baitware, I’d rather offer a better experience at a reasonable price than a mediocre one for free. If it allows me to devote more time to the site and provide a better feature set, some kind of paid model would be an all around win.
After a lot of delay, I think it’s time to go down that road. More details will be provided as I figure them out, but I think the following is a safe high-level picture:
- Most of the site will always be free. From a purely self-interested perspective, I want to be able to hop on someone’s laptop and sketch out a map without having to log in, verify my account, etc.
- Any paid features would be part of a subscription rather than pay-per-use. A number of people have suggested pay-per-print, but that seems rather petty; I’d prefer that people not have to ration their usage.
- I’ll try to align paid features with my cost structure, i.e. the features that cost me more time or server resources to support will be the ones I charge for.
- Everything, including offline map data, will continue to be free for SAR and other first responders.
- I’ll be selling related items like GPS maps and 3D posters as an additional funding source.
So with that in mind, what are some things that might become part of a subscription? Nothing is settled, but ideas that come to mind include large PDF sizes like 24×36, PDF links that last longer before expiring, WMS map access and having elevation data show up by default. I’d also like to integrate non-free map sources (i.e. I need to pay the map provider), and a subscription seems like the logical way to tackle that.
The goal is to devote more energy to CalTopo, not to let it wither while I try to wring money from an ever-shrinking user base – though that seems to be a popular approach, too. Paid features are an experiment, and if they turn out to be a bad idea then I’ll drop them. Changes will be neither drastic or immediate, and you don’t need to worry that PDF generation or some other critical feature will suddenly wind up behind a paywall.
All of that said, I’m open to suggestions.
