TinyGeocoder Has Gone Plaid (Open Source)

It’s been a while since my last post, which honestly is a good thing. That means Tiny has been working flawlessly for quite some time.  I love seeing things “just work,” especially when they are my own personal pet projects.

It seems the service has been working too well though.  And unfortunately the time has come for me to move the service.

I apologize for the predicament this puts some of you in right now as I know we’ve had applications built off our service.  Generally, I’d rather do the “right thing” and let you all have some time to migrate.  The problem though, is that the success of this project has begun to hamper my own projects (and income).

Since it’s inception, Tiny Geocoder has served, at last count, 2 million geocoding and reverse-geocoding queries per month (and growing).  With that kind of success comes pain points.  The biggest pain point is our servers.  We simply can’t keep up with the amount of unique queries which get served every month. Unfortunately, we have some other (profit-making) projects which are of utmost importance to our San Diego based web development agency as well as my (and my family’s) personal needs.  We now have three projects sitting on the same servers and the hammering we’ve been getting lately on the geocoder has been also destroying our uptime and increasing our latency for our other freshly minted (but currently private) projects.

This means our new clients and partners are getting the short end of the stick, and they’re the ones keeping our lights on.

So, because of some extenuating circumstances which put us into a position where we had to make a very fast decision, we’ve tried to do what we could to keep the spirit of Tiny Geocoder alive – namely open-sourcing the code on GitHub.  We wish we could have had more time to let everyone know before we took it down, and we should have taken your email addresses so we could keep in contact about the service, but simply put, there just hasn’t been time nor resources.

Please accept our apologies for the timing and lack of forewarning.  We hope to make it up to you by giving you the code fully and outright.  Other than the fact it’s been migrated to use SQLite, it’s the exact code we’ve been using for years.  Please feel free to download it, commit changes and enhancements to it, and generally improve it for society.

Thank you for all the amazing years, queries, and value that you’ve added to my ego and the service itself.  Please keep the ball rolling and give to others by committing amazing things to the now public code base.

Sincerely,

Nate Ritter

Issues with Geocode.us and Documentation

The past few days were unfortunately fraught with errors because of some issues we’ve been seeing coming from geocode.us and it’s service.  Temporarily, we’ve removed them as a data provider until we can figure out how to put them back on without disrupting service again.

Additionally, we’ve had a (very valid) request for (long overdue) documentation.  At the moment, there is no documentation for what should be returning if things go wrong with a request.  In the past, we would always return an address.  However, there are times when this isn’t always the best idea.  So, we’re starting to document our internal error codes and we’re looking for an easy way to reply without breaking your applications as they are already created.

In the future, we may even need to version our API so we can create new services.  In all, please keep the feedback and comments coming.  As you may already know, we do listen and we do take action on your concerns.

Thank you.

Fixed no-response issues

Thanks to a few of you for pointing out a flaw (and sending in the addresses you were using) in the tiny geocoder recently, we’ve found and fixed the issue which recently crept up where the server would respond with a 200 http code, but have no contents in it’s response.

We apologize for the problems this may have caused and encourage you to let us know if you see any other issues.  You can help us by sending us the address and/or the lat/long that you’re attempting to send in, so that we can find the issue and fix it.

Thanks again peeps. You make the service better every day!

Goodbye 620 Errors

Well, thanks to those of you who were nice enough to send me emails about all the 620 errors we were getting recently, you’ve pressured me into fixing them (at least for regular geocoding). So, today I integrated a new data source, and I’m hoping we’ve solved the majority of the 620 errors that you developers have been getting.

We are still looking into fixing the reverse geocoding 620 errors, so be patient with us there. (see update below)

Please let me know if you keep receiving errors or other issues. I’ll keep looking into making the service more stable and reliable. Your feedback counts.

Cheers.

Update: I just found another data source for both regular geocoding and reverse geocoding, so hopefully that will clear up the 620 error codes for good.  Again, let me know if you run into any more.

Downtime Resolved And Source Discovered

Alrighty, let’s get this whole “downtime” issue out of our vocabulary now. That was a very long and unnecessary stretch and we’re very sorry for the inconvenience. We received scads of phone calls, emails, and tweets regarding the service being down and thankfully (kudos to all of you!), we weren’t scared off by the comments.The thing with computers and networks is that problems can occur. We always try our best to prevent it happening but even when you are using the best computers or fast speed internet like Broadband, problems can still occur. It does make a world of difference to us however when we recieve information from you about what problems we are having and how we can improve on certain things.

Again, you are why we do this.

So, thank you for being so nice when we’ve struggled.

Here’s why we went down:

• It started with some crazy database crashes, which we fixed quickly by increasing the number of connections allowed to the database.
• After quite a few more times of needing to restart the database service and apache… and scratching of our heads… we finally figured out that the reason the database was continually crashing was that we had run out of space on our hard drive.  That crashed the database, did nutty things to the tables, and more.
• So, we added some notifications and enabled some restrictions so that we get alerts now when we get close to capacity.  The nice thing is the table in question is just a data repository – not needed for operating anything – so we just backed it up for posterity, emptied the table, and voila, we’re under capacity and everything’s back to normal.

What we did find out in the process was that we hit 26 million queries.  That’s just crazy.  With this kind of service, having to do this type of maintenance every 1-2 months is doable, but not really on my top list of things I want to do on a Saturday.

Thus, we’ve come to this… we’re considering how to monetize this service so that we can pay for either moving into the cloud or increasing our service at our outstanding host.

If you have any ideas – things you’re willing and wanting to pay for – we’re all ears.  Paying for the service, even in small chunks for the biggest users, would be a huge step up for stability.  “All for one, and one for all.”  We’re for you.

Bring your thoughts … we want to hear them.

Unexpected Downtime

So, as many of you have probably noticed, we had some unexpected downtime in the past 2 days.  We’ve since fixed the issue permanently and shouldn’t see this kind of problem again in the future.

Thank you all for your patience.

Question for App Makers Using Tiny GeoCoder

Thanks to you, our fabulous community of users and developers, we’re coming up on our 20 millionth API query (since we started counting). It’s an amazing thing to watch something like this take off as it has. I’m honestly taken aback every time I think about how many apps and people are using this extremely simple geocoder. So first, thank you.

Now, because you’ve been so amazing as a community, I’d love to get your feedback. I’ve been racking my brain on what I could potentially do to help you more. Tonight I came upon an idea and I wanted to see what you all thought.

The Question for Geo Coder Application Developers

The open ended question is… What else can we build for you?

The more direct question/idea I had was whether or not statistics would be helpful for you? Similar to Bit.ly or Su.pr and their url shortening, I thought perhaps you would like to know things related to your apps geo-coding like:

• Regions, countries, states, etc, are being geocoded the most?
• How often your API key (if we made one for you for trackability) is being used? (how many queries has your app hit us with?)
• What IP addresses are using your API key?
• What regions, countries, states, etc, are using your API key?

That’s just some of the ideas I came up with off the top of my head and some of them might not be useful at all since I would assume you might be tracking this data yourself. But if you liked these and/or had any other ideas that you’d like to see, I’d love to hear what they are.

So? What else can I do for you?  What would be the most valuable stat or service I could produce using this information for you?

IP Address to Location: Geolocation

Of course, address geo-location is just one way to geo-locate someone. Other ways of geolocation can be done using IP addresses, and although it’s never perfect it can be helpful in certain circumstances (like customizing your website for localized searches and visitors).

If you’re looking to find a location from an IP address, I’ll tell you right now that IP2Location is the best in the business.

I’ve implemented these guys APIs in the past to help customize and localize my users experience on websites and applications. Just start thinking out of the box about what you could do with something like that.

Cheers. Happy geo-coding.

New JSONP (JavaScript Callback) Support

I really love great suggestions. A few days ago David Evans from CanTab sent me an email suggesting an additional feature be added to Tiny Geo-Coder. He was kind enough to send along an example of what the new API call might look like and what the response might look like. It was so simple, I couldn’t resist.

This evening, I implemented JSONP support into the Tiny Geo-Coder API (JSONP in Wikipedia). So, now, as the documentation explains, if you send a querystring variable titled “callback” with the value being whatever function name you’re wanting to use, you’ll receive it back, wrapping the geo coordinates (or location, if you’re doing reverse geo-coding) in your function.

David explains the use case for this best:

The beauty of this is that it allows you to use the API in AJAX apps without running into problems with the same-origin policy.

So, there you have it. Go run off and create some awesome AJAX apps and report back to me what you’re doing so I can give you some props here on the blog.

Cheers,

Nate Ritter

13 Million API Queries Later …

As you may know, I created this nice little service back in March of 2008 but didn’t release it publicly until around November 2008 because I didn’t think it was that valuable. Boy was I wrong. Apparently it’s been valuable over 13 million times to many of you who are reading this today since early December ’08.

That’s freaking amazing. I honestly still laugh at the simplicity of the app and think it’s hilarious that people are even using it. But, at the same time, I fully understand how much it’s needed now.

In case you didn’t get that, it’s been about 6 months and we’ve had 13 million queries on our API (if you can even call it that). That translates to about 3,000 queries per hour or 50 queries per minute. I’m actually amazed we haven’t gone down more often or run into the infamous “620″ error more. I mean, I know it’s simple, but still. I’ve never personally created anything (by myself) that’s gotten this much use before.

Regardless of how that gets me all up in an excited tizzy, the real thanks goes out to you! Thank you for using the service. Thank all of you who have given feedback and suggestions. Thank you for linking to this site, writing stories about it, blogging about it, and spreading the word. Thank you for thinking it’s useful and using it. You deserve 13 million thank you notes.. but don’t worry, I won’t spam you with that.

Please, as always, let me know what I can do to help you with your app. Also, please don’t be shy in letting me know what you’re doing with it. I would love to start highlighting people’s projects and companies who are using the service. I’d love to share the “thank you” with some link love and press for ya.

Thanks again! Keep on geocoding!

Nate Ritter, Web Chef