How It Works

Basic Overview

DataSprocket is a web and mobile-friendly application for creating, editing/managing databases, as well as a cloud based service for sharing your data with your websites and web/mobile apps via a very simple API that should be accessible to developers of all skill sets.

If you are a developer, you can find programming classes/objects, and documentation for many languages in the "Connectors" section, below.

If you are relatively comfortable with working with HTML and CSS but not with programming, we have provided a number of connectors for popular platforms that, once installed, will do the heavy lifting of allowing your website or app to communicate with DataSprocket. If your platform is a subscription service such as SquareSpace, Weebly, Wix, or WordPress.com, you can use the Javascript connector and the copy-and-pasteable code samples to allow your site to access and display your DataSprocket data. If you come up with some snazzy templates and would like to sell them to other DataSprocket users, please contact us and we will arrange making these available within the in-app Sprocket marketplace.

If you are neither a developer nor somebody comfortable with HTML/CSS, we will soon have premade templates available for purchase for a small fee that, along with the connector for your platform, will provide beautiful displays you can use to display your data without any programming at all beyond entering something like [ds template=14] into the page in the place where you'd like this template displayed.

How DataSprocket Works (Developery Stuff)

The following is an overview of how DataSprocket works, and how you can interact with it within your websites/apps …

Communication to DataSprocket is made via conventional HTTP REST requests, and the responses received (which require authorizion) are JSON encoded, and this is the same no matter what connector is used. Authorization to your data is based on the:

  • Requesting IP address
  • Having access to the unique site ID assigned to each DataSprocket account

The basic steps needed to get your data to display on your site are as follows:

  1. Within your DataSprocket settings, take note of your site ID – you'll need that later. Add the IP addresses of your sites you'd like to permit access to.
  2. Within DataSprocket, access the Connector Settings for the table(s) you'd like to display on your site, enable connector access. This is found within the "Settings" menu for that table.

For the Javascript Connector

  1. Copy and paste the code included in the "Javascript Connector Code" section, below, into your website as instructed.

For All Other Connectors

  1. Install your connector (connector specific installation instructions can be found on the respective Github page linked to from this page).
  2. Provide your connector your site ID into its configuration. The location for this config will vary, see the instructions included on the Github page for further info.
  3. Decide whether you should cache your data locally (see information about this, below). If you do, enter this into your connector's config as well.
  4. Find copy-and-pasteable code for requesting data from DataSprocket in the respective Github page.

Should I Use the Javascript Connector?

The Javascript connector will work on all platforms without having to install anything at the server level. In most cases this is the best idea if the following does not apply to you:

  • There are sites other than yours that are available via the same IP address as the one assigned to your site (if your host is a shared hosting provider such as GoDaddy, HostGator, Bluehost, or MediaTemple, there is a good possibility that your IP address is shared), and…
  • Your data is proprietary and/or highly coveted, and you don't want to make it easy for other sites that happen to also be assigned to your IP address to have easy, direct access to read your DataSprocket data which you have decided to share with the public (and note, it is not possible for anybody to modify your data without having your DataSprocket account/login info, and this does not apply to tables which you have not explicitly enabled connector access to, nor, again, anybody that does not share your IP address). The reason that this is possible is because, due to how Javascript works, it is possible to use a web browser to intercept a Javascript connection and, in this case, obtain your site ID. Of course, anybody that really wants to steal your data can simply copy and paste your information from your site, so this concern is only relevant to a small number of use cases where, for example, data is sold. With the other connectors, your site ID will be encrypted in transmission, so that this is not a concern. Or…
  • Your tables are very large and/or you want to maximize every last millisecond of performance. The other connectors allow local caching of data for quick and easy access.

In short, the Javascript connector is great for data you wish to make completely publicly available, is great for tables that aren't enormous in size, and is the quickest and easiest way to get your website/app working with DataSprocket.

Caching/Performance

As noted above, if you wish the maximum possible performance, it is wise to look at any other connector other than the Javascript connector, as local caching of data is supported.

Without local caching, your data will be fetched in real-time directly from DataSprocket. Otherwise, with caching enabled, at a time of your choosing (for example, each page load, or every x minutes), the connector will quickly contact DataSprocket to see if any data has been changed. If it has, your cache will be regenerated. If it hasn't, your local cache will be used.

A single line of code is used to trigger the check to see if the local cache should be updated, you could run this each time a page is loaded, manually, or via an automated cron job that runs at regular intervals.

Relational Database Support

DataSprocket supports, and makes heavy use of relational database design principles. Relations between tables are defined within the application, and when sharing your data with your website/app these relationships are respected. Unique IDs are assigned to each table row should you wish to make use of these primary keys.

Searching

Your app or website may want to support displaying a subset of the data within a table, or conducting live searches. If you'll know in advance what sort of data you'll want, you could have your connector request the results of a saved search you have defined within your DataSprocket account. Otherwise, you can run your own searches locally based on the entire data set within a table.

Sharing

You can share your data with as many users as allowed by your service plan. If needed, your service plan can be upgraded at any time without causing any disruptions in access to your data. Creating a DataSprocket account is free, and users can be invited to an infinite number of DataSprocket projects so long as the project owner's service plan supports this.

Security

Data transmitted within the DataSprocket application is SSL encrypted, and data requested by your website/app from DataSprocket is likewise encrypted providing you aren't using the Javascript connector (see above).

With this encryption, and access/authorization being two-factor in requiring both the unique site ID and an IP address that has been granted explicit access to your data, DataSprocket is secure. However, we do not recommend using DataSprocket with super sensitive information such as social security numbers, credit card numbers, or medical records, as you might be putting yourself at legal risk in doing so (in these cases, with PCI compliance and HIPAA laws), particularly if you wish to share this data with a website or app of some sort. We are not responsible for any damage which might be caused in doing so.

Support for Large Databases

DataSprocket is designed to support tables/databases of all sizes. It supports infinite scrolling (loading data in small chunks), lazy loading of images, internal caching, and a number of other techniques to ensure that performance is not an issue with managing large amounts of data.


Supported Field Types

DataSprocket provides support for a wide variety of different field types to ensure that your input is quick, efficient, and accurate.

Special Field Types

  • Toggle on/off switch
  • Autocompleting field
  • Check all that apply
  • Number/numeric range
  • Text box with editor
  • Date/Time with calendar based input
  • Location (uses Google Maps to pinpoint latitude/longitude)
  • Email address
  • Website URL
  • Image uploads (with automatic image resize options)
  • File uploads
  • Encrypted passwords
  • Color picker

Adding Field Types

Please contact us if you'd like to suggest a field type that DataSprocket should support.


Connectors

Coming Soon

These connectors are in the final stages of development and will be available shortly…

Javascript

Copy-and-pasteable code for all hosting environments, platforms, and browsers
DownloadView Source Code/Docs

WordPress

For all WordPress sites/versions, excluding sites hosted on wordpress.com
DownloadView Source Code/Docs

PHP

Supports data encryption and local database caching. Requires PHP 5.2 or greater, or earlier versions of PHP with the JSON extension enabled
DownloadView Source Code/Docs

Planned

A commitment has been made to develop these connectors, development will begin shortly…

Phonegap

Compatible with all versions and devices supported by PhoneGap

Ruby on Rails

Ruby on Rails Gem plugin

Joomla

For Joomla 2.x and higher

Request a Connector

Please vote on which connectors you would like to have built. Your feedback is very important to us. Is the connector you are most interested in not included in the list below? Please contact us to let us know.

Expression Engine
0 votes
Drupal
0 votes
PyroCMS
0 votes
NodeJS
2 votes
Python
3 votes
.NET
1 votes

Make Your Own Connector

Information available shortly…


Personalizing/Sprockets

Sprockets

Sprockets consist of plugins that provide additional DataSprocket features, visual templates for your websites or applications, pre-made databases, or personal customizations for your projects or business.

Sprockets (that aren't free of charge) can be purchased from within your DataSprocket account within what we are developing to be a marketplace for Sprockets. In addition to providing new features, we have developed DataSprocket to provide a wide range of project-specific customization for automating tasks, generating custom reports/views, and more. Please contact us if you have any questions about what the Sprocket architecture can do for your business, or to request a quote.

Additionally, we are hoping to develop an extensive selection of templates for presenting data in all sorts of interesting ways, and sparking the creativity of our users to not only develop stunning presentations for their own projects, but to give them the opportunity to sell their templates (which consist of HTML, CSS, and optional Javascript) on this marketplace to other users!

Please let us know if you have any ideas for developing specific Sprockets. We want to make DataSprocket better and more flexible to as wide of range a user as possible.

Branding and/or Reselling

DataSprocket can be branded for your organization or business via a special "Sprocket" which supports accessing DataSprocket from a subdomain of your company domain, having your logo visible, as well as the overall color palette of the application being modified as desired.

If you plan to use DataSprocket with a number of different projects, multiple service plans are available which will allow these multiple projects to be branded under the umbrella of a single parent company, as well as to access these projects from a single login. If you wish to develop an application of your own that uses DataSprocket, using the "Unlimited" service plan you can support an infinite number of projects, and when paired with the special branding Sprocket, your customers will not even be aware of DataSprocket's presence.

Branding requires a one-time installation/setup fee of $200 (which does not affect your month-to-month costs), and this Sprocket can be purchased at any time from within DataSprocket.

Virtual Private Servers

Does your website or app get a ton of traffic? If so, DataSprocket can be run on your own managed VPS (virtual private server). We will monitor and maintain your server using Amazon's EC2 environment and add computing resources to it as needed, giving you the peace of mind that as your business continues to grow, DataSprocket will not be a bottleneck. Please contact us for a quote, or for any questions pertaining to this special Sprocket.

DataSprocket's design of separating the data backend from the frontend also allows you to have a variable number of backends serving data to a variable number of web servers. Additionally, it allows you to consolidate resources so that the same backend powers both your mobile apps, as well as your website/web apps.

For Hosting Companies

If you are a hosting company, we would love it if you would make our DataSprocket connectors available to your users. Please contact us to arrange referral discounts for your users.


Support

Contact Us

Please contact us with any questions or feedback, we will respond promptly…

Hire a Developer

Need help from a developer? Please contact us, we'd like to introduce you to one. Are you a developer interested in helping other DataSprocket users with their integrations? Please contact us, we'd like to meet you!