This project has moved and is read-only. For the latest updates, please go here.

Entity framework, Update triples, Working with graphs

Dec 13, 2013 at 12:07 PM
Hi,
I have a few questions:

1 Does brighstardb support working as a quad store?

Entity framework api

2 Considering an update has 2 parts: triples to add/ triples to remove. When saving changes can i get the add triples, remove triples.
Something like dbcontext.SaveChangesInMemory(IList<Triple> adds, IList<Triple>removals) and then I could use those lists do do whatever I want.

3 When saving changes/ or running queries can I specify the graph to work with?

4 After forming a query can i get the query that will be sent to db without actually sending it?
For example: persons.Where(p => p.Age >30).ToListAsQuery() could result in a sparql string query that I can use further. I could use this when creating more complex queries by combining EF generated queries with hardcoded ones.

5 Does it support deleting triples base on a criteria?
DELETE WHERE { ?person foaf:givenName 'Fred';
                       ?property      ?value }
Dec 13, 2013 at 1:54 PM
feugen24 wrote:
Hi,
I have a few questions:

1 Does brighstardb support working as a quad store?
Yes. It is a quad store.
Entity framework api

2 Considering an update has 2 parts: triples to add/ triples to remove. When saving changes can i get the add triples, remove triples.
Something like dbcontext.SaveChangesInMemory(IList<Triple> adds, IList<Triple>removals) and then I could use those lists do do whatever I want.
Not at present but it could possibly be added.
3 When saving changes/ or running queries can I specify the graph to work with?
Yes, see http://brightstardb.readthedocs.org/en/latest/Entity_Framework/#graph-targeting
4 After forming a query can i get the query that will be sent to db without actually sending it?
For example: persons.Where(p => p.Age >30).ToListAsQuery() could result in a sparql string query that I can use further. I could use this when creating more complex queries by combining EF generated queries with hardcoded ones.
No not at present.
5 Does it support deleting triples base on a criteria?
DELETE WHERE { ?person foaf:givenName 'Fred';
                       ?property      ?value }
Not in Entity Framework, it only works on entities, But you can use SPARQL Update to do this. See http://brightstardb.readthedocs.org/en/latest/RDF_Client_API/#update-data-using-sparql

I think the extension points you want in (2) and (4) are good things to add. One possibility for doing this would be to allow you to pass in an IBrightstarService instance instead of creating it internally, that way you could implement a wrapper around the existing implementations that just trap the calls to ExecuteTransaction and ExecuteQuery. That might be more elegant than hacking in events for every possible extension point someone might want...
Dec 13, 2013 at 2:40 PM
For 5 I meant it for entities but gave poor example so it could look like:
datacontext.Persons.DeleteWhere(p => p.Age >30);

It would be nice to have this features(2,4) in one of future versions, since they would give more flexibility. I would really like to use the EF api but, i don't think i can use the api without them because i need more control for certain scenarios.

Also in the docs:
The code will send a set of SPARQL update commands in a single request to the store. If the store does not implement the processing such that the multiple updates are handled in a single transaction, then it will be possible to end up with partially completed updates.
I'm not sure i understand why it's a list of updates instead of one update that should be atomic in most stores. In the end it is adding/removing a list of triples.
Dec 13, 2013 at 3:46 PM
feugen24 wrote:
For 5 I meant it for entities but gave poor example so it could look like:
datacontext.Persons.DeleteWhere(p => p.Age >30);
Hmmm - that would be a more major extension because:
a) it doesn't translate into the simple transaction API without first having to execute the query (which means it ends up not being atomic). Which would mean having to always use the SPARQL Update route rather than being able to use the more efficient ExecuteTransaction for BrightstarDB connections.
b) It would mean having to add more LINQ processing to flip a query into an update

It is possible I think, but it would require quite a bit of effort, and for BrightstarDB connections it would mean sacrificing some performance so I'm not keen to do it just yet.
It would be nice to have this features(2,4) in one of future versions, since they would give more flexibility. I would really like to use the EF api but, i don't think i can use the api without them because i need more control for certain scenarios.
I'll see what I can do about adding it in 1.6
Also in the docs:
The code will send a set of SPARQL update commands in a single request to the store. If the store does not implement the processing such that the multiple updates are handled in a single transaction, then it will be possible to end up with partially completed updates.
I'm not sure i understand why it's a list of updates instead of one update that should be atomic in most stores. In the end it is adding/removing a list of triples.
Well it would be nice if you could do that in SPARQL Update. But it needs to be DELETE DATA { triples }; INSERT DATA {triples} which is two separate SPARQL Update commands. There is a DELETE {triples} INSERT {triples} WHERE { patterns }, in SPARQL Update, but AFAIK you can't have an empty WHERE clause, and I think you can only target one graph or the default data set - its not possible to target a defined subset of graphs. I did try to find a way to format it into a single command, but couldn't quite get a method that would work for both targetted graph updates and general store updates. so I gave up and just went for a list of commands. The code is all in BrightstarDB.Client.SparqlUpdateableStore.cs - if you can find a way to make it execute single SPARQL Update commands for the different combinations of graph targetting that would be a very welcome pull request!
Dec 14, 2013 at 10:24 AM
Edited Dec 18, 2013 at 9:49 AM
I think 5 is not that important since i can always write a custom query to do so.

Related to atomic updates i think you are right...i had the impression that sparql specs have delete data + insert data in one command, so i think it's up to the store implementation as you said. But also if the EF api is flexible enough then we could write custom implementation as needed (i will explain below).

I though a bit about my previous suggestions(2, 4, atomic update) if they are ok, and how I would like to use them, and i'm thinking each triple store uses sparql as standard but add it's own tweaks to allow certain scenarios. For example an update request can return nr of modified triples, execution duration, and so on. So the more I can control EF api the better.

So i see the flow of a query like so:
EFQuery -> Sparql Query -> HttpRequestToStore -> StoreHttpResponse -> SparqlTriplesResult -> MapTriplesToEntities -> Entities

For a save:
Datacontext.SaveChanges -> Added/removed triples -> HttpUpdateRequestToStore -> StoreHttpResponse

What would be nice is to be able to hook into this pipeline at any point, override existing implementation and also pass a parameter so decisions can be made (in the overriding implementation) based on that. In other words each step of the pipeline executes in a context that has some info, our parameter.


Also i would like to write custom sparql queries that can be run on the dbcontext and then map the results to entities. Or some way to map the results of a custom made sparql query to entities (since context has mapping info).
Dec 18, 2013 at 10:21 AM
I looked a bit through code to see what could be done to add more control but not involve too many changes. (the above suggestion would be nice but harder to implement)
As far as I understand at this point I noticed the following:

BrightstarEntityContext is to be derived by an application to use as dbcontext for it. Internally it creates an "IDataObjectContext"(dnr, rest, embeded) based on connection string. In case of "DotNetRdfDataObjectContext" it needs to resolve update/query processors also from connection string.
The problem I see here is that there is this static connection that hard codes all dependencies.

I would suggest:
  • make BrightstarEntityContext have an overload that accepts an IDataObjectContext instead of connection string
  • make DotNetRdfDataObjectContext public so I can create an instance of it
  • add an overloaded constructor DotNetRdfDataObjectContext(ISparqlUpdateProcessor, ISparqlQueryProcessor)
In this way I can create my own processor and removes configuration dependencies.

It is very helpful to run all unit tests against a dotnetrdf store/endpoint instead of embeded one. At this point I had to do some hacks to make this work, but now I can see that I have 4 tests that fail and can investigate further; it also gives me more of confidence in my EFapi to dnr-store connection since tests pass.
  • consider changing unit tests to use BrightstarEntityContext instead of embeded one, with config setting for embeded by default. (or some way to let me use a dnr connection where is appropriate, so excluding bsdb specific tests)
I could help on the implementation of this part if necessary.
Dec 18, 2013 at 4:57 PM
I was thinking along the same lines as you - it would also have the effect of making it easier to use dependency injection frameworks with BrightstarDB. Ideally I should be running all our unit tests against both an embedded context and also against a DNR context so that it exercises the DNR wrapper code a bit more, so doing this would really be a help as I could then set up some parameterized test cases.