Distributed Asynchronous Composable Resources

Imagine a data table, where each row is an item and each column is a property.

It might look like this:

The table is a representation of a collection of objects, each with several properties.

Using JavaScript notation, they would look like this:

[
    {
        url: 'http://example.com/1',
        name: 'Item One',
        published: '2015-09-10',
    },
  {
        url: 'http://example.com/2',
        name: 'Item Two',
        published: '2015-09-11',
    }
]

An abstract definition of the object, using Polymer’s notation, would look like this:

{
    properties: {
        url: {
			type: URL
		},
        name: {
			type: String
		},
        published: {
			type: String
		}
    }
}

You might notice that the published property is represented as a String, when it would be easier to use as a Date object. To convert it, you could add a “computed property”: a function that takes one or more existing properties as input, and outputs a new property:

{
    properties: {
        url: {
			type: URL
		},
        name: {
			type: String
		},
        published: {
			type: String
		},
        publishedDate: {
			type: Date,
			computed: function(published) {
            	return new Date(published)
        	}
        }
    }
}

From this definition, you can see that the publishedDate property has a dependency on the published property: any computed properties should be updated when any of its dependencies are updated. In this case, when the published property is updated, the publishedDate property is also updated.

This is fine when the dependencies are all stored locally, but it’s also possible to imagine data that’s stored elsewhere. For example, this object might have associated metrics data counting how many times it’s been viewed:

{
    properties: {
        url: URL,
        name: String,
        published: String,
		publishedDate: {
			type: Date,
			computed: function(published) {
            	return new Date(published)
        	}
        },
		viewCount: {
			type: Number,
			computed: function(url) {
            	return Resource(url).get('json').then(function(data) {
                	return data.views
            	});
        	}
        }
    }
}

The Resource object used above is a Web Resource, part of a library I built to make it easier to fetch and parse remote resources. If it helps, an alternative using the standard Fetch API would look like this:

return fetch(url).then(function(response) {
    return response.json()
}).then(function(data) {
    return data.views
});

In either of those cases, the data is being fetched asynchronously, and a Promise is returned. Once the Promise is resolved, the viewCount property is updated. If this property was bound to the original table, you would see the new values being filled in as the data arrives!

Implementations

I talked about this kind of thing at XTech in 2008, illustrating the object as a Katamari Damacy-style of “ball of stuff”, being passed around various different services and accumulating properties as it goes.

vege-table is an implementation of asynchronous composable resources: it’s an interface for adding computed properties to a collection of items and fetching the data asynchronously into a table.

Talis’ data platform had a similar feature, where results from a SPARQL query could be augmented by passing each result through another data store, matching on identifiers and adding selected properties each time.

The SERVICE feature of Wikidata’s SPARQL endpoint is also similar: it takes an object in each result and passes it to a specific service, assigning the resulting data to a specified property.

In a Google Sheet, adding a column/property using the IMPORT* functions will fetch data from a remote resource, inserting the value into the table when it arrives.

In OpenRefine, remote data can be fetched from web services and added to each item in the background.

Are there any others?