AirTable’s API limits the requests 5 requests per second per base and suggests using a caching server to get anything more than this.
We have set up a low-cost GCP f1-micro instance to act as this caching server so our site can handle bursts of traffic that exceed 5 requets a second without being locked out for 30 seconds by airtable.
All AirTable libraries that allow you to change the endpoint URL should work with our caching endpoint
The api can be accessed publicly https://api.vacfind.org
At present our caching server requires no additional authentication as it just forwards and caches requests to airtable’s API. if you have an airtable base ID and API key, you are good to go.
Here are some general-purpose ideas for ways to mitigate this:
There isnt anything special about our caching server that limits it to being used for just our own airtable bases or API keys, While untested, its entirely possible for anyone to use their own base ID and APIkeys to read their Airtable bases and have the results cached for much faster delivery.
Currently the caching server is not explicitly set up to support writing data to your airtable databases, however, it should work. If you need this and run into issues that require configuration on our side, please contact us.
This API was initially set up for our own use at VacFind so it wasnt configured with much redundancy or loadbalancing in mind. That said, the capability exists so if higher availability is something you need, contact us.
We are just some students with a little free time trying to help as much as we can. We are unale to be available 24/7 to respond to issues or
The caching server uses Caddy to provide super easy HTTPS. Caddy also has the capability to provide load balancing should
By some measures, Varnish is one of the fastest caching servers there is. It is also able to serve older content from the cache in case your actual API goes down as opposed to showing no data or an error. While our caching is not currently set up for this, we plan to set this up soon.
Because varnish doesnt support HTTPS, we use stunnel as a HTTPS client to create secure requests out to Airtable’s API when varnish needs to fetch records direct from airtable.
The API has some basic monitoring set up to alert us if there is downtime.
While we will do our best to host this for free/out-of-pocket, we ask that projects that have available funds, especially those that heavily use our cache, contribute what you can to help us provide this caching ability for smaller projects that arent able to afford it.
Donations can be made through our OpenCollective Page and are tax-deductible.
While we are just some students with a little free time trying to help as much as we can, we would love to help you make the most of our caching solution so you can focus on building and maintaining your apps.