vege-table: the data table that grows, with leaves

Sensors are being deployed all over the world (and beyond).

Big companies (Google, IBM, Wolfram) are positioning themselves to be the repository where sensors store their data.

Other companies are building platforms for applications to make use of that data in real-time.

There’s a piece missing: it should be possible to query those data stores to build up a snapshot of information, then document and publish the collection of data (and the harvesting process) for others to read and explore.

vege-table

vege-table, a custom HTML + JS element built using Web Components + Polymer, provides an interface for collecting, describing and publishing such a set of data.

Note: this is an initial release and a work-in-progress. It works, but only just, and not in all web browsers, yet.

How it works

Firstly, seed-harvester imports an initial collection of items (which may be as simple as a list of identifiers or URLs) from CSV, JSON, or a JavaScript function that fetches the initial data set.

Secondly, leaf-builder provides an interface for adding leaves (properties; computed or otherwise) to each item of the data set. The leaf can be any standard data type, including an asynchronous resource. Once a leaf has been attached to an item, the data added can then be used to build further leaves.

Thirdly, vege-table itself extends HTML tables to present the collection of items, generating a row for each item and a column for each leaf. The data table is paginated, sortable, filterable, and includes footer rows that summarise columns using facets where appropriate.

Finally, tuber-storage stores items in a database (currently PouchDB) as they are updated.

Once all the leaves have been added, the data collection can be published by exporting the table description and data files, placing them in the same folder as the main index.html file, and switching off the database.

vege-table and its components/dependencies are available on GitHub, and installable with Bower.