I run ifconfig.io which now gets just over a billion hits a day. It is basically an echo service, and it just parrots back what cloudflare tells it. But since I run it on linode, it costs me a whopping $40 dollars a month for 35 billion returns a month. Using workers would cost me several thousand dollars.
It's a nice solution. Why log into your alt account to crap on it?
I'm trying to learn to take it as the highest form of compliment - when hackers care enough to hack on one of my ideas, that means it has to be cool.
https://blip.runway7.net https://github.com/runway7/blip
I lost track of the consumer apps I've used this on and still haven't received a $1+ bill.
"time_zone": {
"name": "America/Chicago",
"abbr": "CDT",
"offset": "-0500",
"is_dst": true,
"current_time": "2020-07-25T06:10:16.945136-05:00"
}Moment.js is probably the most prevalent time and date library.
Programmatically populating the response body, as in the Cloudflare worker example from the post, is better than going to the origin just to echo some headers back in the response. To me, something like Fastly's VCL config language is even simpler. It directly executes on every CDN edge node worldwide upon request.
For example, I just whipped this up on Fastly using VCL. It returns GeoIP as json data for your IP at the root path:
Or if you want a particular IP, just append it to the path:
http://geo.zombe.es/2a04:4e42:600::313
You could do the same via query params, headers, etc. Have URL endpoints that only return some of the data, and so forth.
The VCL syntax gets a little gross when you handle quoting strings and assembling json and testing if the string is empty, but it gets the job done.
Of course what you might want from GeoIP data may not be what you get. It's really kind of a useful kludge that gets treated sometimes as a panacea.
This dataset right now thinks that I'm about 5 miles east of my location, but when subnets are repurposed it could be much more significant. And the data sources are always changing, so who knows what it will think tomorrow.
"client": {
"conn_speed": "broadband",
"conn_type": "wifi",
"proxy_description": "vpn",
"proxy_type": "hosting"
conn_speed: https://developer.fastly.com/reference/vcl/variables/geoloca...
conn_type: https://developer.fastly.com/reference/vcl/variables/geoloca...
proxy_desc: https://developer.fastly.com/reference/vcl/variables/geoloca...
proxy_type: https://developer.fastly.com/reference/vcl/variables/geoloca...
conn_type is interesting to me, I'm not sure how you would distinguish wifi vs. wired based on HTTP header data.
vcl_recv and vcl_error are the important bits, the rest is VCL boilerplate from https://developer.fastly.com/learning/vcl/using/ for unused subroutines.
It's very borderline from them (at least ethically)
The response time averages 10ms and is free, granted I'm not sure if there is a ToS or anything attached to this endpoint, I've only found non-conclusive discussion here https://community.cloudflare.com/t/what-are-the-terms-of-use...
I'd advise you to just create a Cloudflare Worker if you want do this. You'll get more reliability etc. and Workers have a very generous free tier.
It's kinda like a construction firm using a supercar to haul materials and then bragging about their efforts to make cosmetic repairs cost-effective... maybe you could just not use an overpriced car to begin with and then damage won't be an issue?
I got sent a (frustratingly incorrect) bot reply to a ticket and a reminder that Enterprise / Business / Pro customers are priority (in that order) even though I pay for Workers. It has been an uphill battle to get someone to take a look at the ticket so far. Thankfully, we haven't gone to production yet, but as a consequence, now need to plan to add mitigation in scenarios where Workers blacks-out our traffic (but Support can't be of immediate help because "free customer").
[0] https://community.cloudflare.com/t/workers-and-sub-requests/...
How is it that you prefer to engineer those workarounds instead of just paying $20/month? It surely can't be worth in engineering hours and ongoing maintenance of hacky workaround code?
If it isn't clear, there's an issue of trust here, not money. And frankly, a fallback shouldn't be much of an issue since fly.io, AWS Lambda@Edge are like-for-like swaps, so we'd still be "serverless" just on a different yet similar platform (albeit costlier).
If the exact location is important geoip is not accurate enough anyway. Forwarding to regional sites automatically is just annoying when it doesn't work properly or someone is traveling abroad.
However, there's a lot of use cases that semi-accurate geo-locations make sense. The first of which is analytics. If I'm a marketing person at a SaaS company, I want to know where my customers physically are if possible for a variety of useful reasons.
Good quality estimates for location also help with security and compliance use cases as well. If a user logs in from a new country on the opposite of the world, you can flag that and take whatever action you want whether it's to block them, fire off an email to the person who owns the account or whatever.
I hope you're factoring in error rates. Because right now Google, with its billions of dollars to spend on geolocation, tells me that my laptop is in Albuquerque, and my phone is in Los Angeles. Neither are within 500 miles of either.
What reasons? What makes you feel entitled to that information if not volunteered?
Of course, you can just get around this with a VPN.
There are many many cases where it’s relevant. Others have mentioned prepopulating or ease of use aspects. But you’re making big assumptions that the client has an interactive user at all. It’s important where you don’t own the client or there is defined behavior, say at a protocol behavior, that must be respected. Moving earlier than that there are cases where you need to route traffic before any sort of connection is even established.
I suspect you’re also underestimating geolocation precision and accuracy. Free datasets will get you the right state or city say 90-95% of the time. Cheap datasets get you to the right town or post code 95-97% of the time. Expensive or bespoke multi source datasets will get you reasonable post code, neighborhood, or even household & address accuracy. Think of cross correlating order histories per physical address, device fingerprints, and IP addresses/triangulation.
https://arstechnica.com/tech-policy/2016/08/kansas-couple-su...
https://www.theguardian.com/technology/2016/aug/09/maxmind-m...
https://mashable.com/2016/08/11/ip-addresses-kansas/
ISP perspective here: Geolocation by granular /24 to /20 sized block of ipv4 space is often wildly inaccurate on a regional basis. It's entirely possible for the ARIN registration (used by maxmind) to be a street address in Seattle, but serving end user ISP customers a 4.5 hour drive away in a far eastern corner of WA state.
I've been meaning to make automatic Github releases for it.
https://www.united-coders.com/christian-harms/detailed-perfo...
It's also free and fast but with services like these I always wonder how long they manage to stay free.
I had good success with maxmind's free database. I can't find tooth results for accuracy but my sampling was pretty good. They also have an incentive to keep it.
6 million requests per month is only 2.2 requests per second, a raspberry PI could do this (technically).
Now I have been thinking about opening it up for more people because while there are a variety of these services out there, one more does not hurt — it already exists, and works, so who knows, maybe more people would like to use it. The code is open, and access logs go to /dev/null; I could probably add a read-only SSH user for people to confirm that for themselves.
Geolocation could easily be added to it, but then comes my question: is there any use-case for geolocation APIs that does not involve tracking users for shitty purposes?
I was excited about adding this feature because I am using pure NGINX for it, and it was a fun learning experience, but I asked myself that question when I started writing the documentation for the website, and I still do not have an answer. Marketing material for other APIs that offer geolocation usually have user tracking as a selling point.
Personally, I have no use for geolocation, and if all use-cases involve tracking without consent users, and breaking their privacy, I want no part in that.
I work with an event org with a fairly common name, and another city in our region has an event with the same name. For some years, we’ve outranked them for major keywords and getting their confused customers was becoming a headache for us.
So now if our website thinks you’re in the other city, a little banner appears above the content asking if you’re looking for the other site, and offers the link.
This has saved us ~10 phone calls per day in season. Before adding this link we all got to the point of just doing their customer service for them and helping people buy tickets because it was less drama than trying to get them to call the right people.
But in any case, your response does tell me that there are other things that can be done with that information, so maybe I should focus on the IP thing, and have the geolocation features kinda hidden away in the documentation pages.
Use my GEOIP and longitude and adapt your CSS to show me your content in nightmode when it is night time in my Longitude. That would be cool.
When I build websites, I usually prefer using `prefers-color-scheme: dark` in CSS, so the user can decide.
[1] https://developers.cloudflare.com/workers/reference/apis/req...
[2] https://docs.aws.amazon.com/AmazonCloudFront/latest/Develope...
[3] https://cloud.google.com/appengine/docs/standard/go/referenc...
There's nothing stopping a script-kiddie from thundering away at the Serverless endpoint resulting in an unexpected and quite high billables [0].
As for Workers specifically, Cloudflare's rate-limiting plan makes the whole thing 10x expensive at $10 for 2,000,000 good-requests [1] + $1 for 2,000,000 Workers requests. Other cloud providers I don't think fare any different.
[0] https://community.cloudflare.com/t/how-to-protect-cloudflare...
[1] https://support.cloudflare.com/hc/en-us/articles/11500027224...
> on average the response took somewhere between from 200ms to 500ms
I'm getting latency of 66ms to 126ms with some simple Java code running on AWS Lambda using provisioned concurrency. I find the latency is just fine for most use cases.
What's the point of using Lambda if you have to provision capacity for it?
You pay a premium on the compute because it's able to scale down to zero (so in theory, it works out cheaper anyway), but in reality it only works well when you do everything in your power to have it not scale down to zero.
Furthermore, Maxmind adds non-factual entries to the database so they can identify it uniquely as their work. Map makers do something similar: https://en.wikipedia.org/wiki/Trap_street
I realise this doesn't directly answer your question but I guess one reason might be that they already provide a free version so releasing the paid version would just get you in strife?
IP to country is fairly easy and I open sourced all the scripts and the database itself [0].
But IP to city is much harder, I'm not actually sure it's viable for anyone to do that without relying on some other 3rd party service.
I'd be very interested to hear if anyone knows how to pull that off in an open sourceable manner.
That law will affect open source solutions just as much.
> Aggregate and anonymous data is exempt from the CCPA, unless it is in any way re-identifiable. https://www.cookiebot.com/en/what-is-ccpa/
Would this be cheaper than running these services?
easier would be and thats just a thought: You could build a go/c++/java app which contains that database pre compressed in some arbitrary better format than just 2 floats, get it down from your original 34gig and just keep it in memory.
Like if you know that 99% of all ips in a certain range are from US, then only store the 1% as a list.
I guess it’s ok for a basic consumer website, although it’s not exactly equivalent to ip geocoder databases/services - they allow passing ip addresses as part of geocoding request.
https://developers.cloudflare.com/workers/templates/pages/co...
AFAIK, an unresolved problem is a proper geolocation service - at least for city level resolution - for mobile IP addresses. There are some services in this field (digital element), but they are very unreliable.
It doesn't seem to have the same problem with Hawaii for some reason, they're all just US from what I can see.
Nginx + GeoIP2 module sets a HTTP request header and proxies the request to the golang app.
The go app does a lookup from redis, based a combination of country header and some url params and responds back with a redirect header.
Hosted it on a t2 medium in EC2 and I have seen it easily handle ~1500 requests / second without any issue.
"Application error An error occurred in the application and your page could not be served. If you are the application owner, check your logs for details. You can do this from the Heroku CLI with the command heroku logs --tail"