GH-3044: HashJoin iterator construction no longer eagerly builds hash probe table.

[Aklakan]

GitHub issue resolved #3044
Pull request Description: Timeouts in QueryExecDataset were postponed until a so-far eager hash probe table construction completed because of locking. This PR proposes to initialize the hash probe table lazily such that the iterator construction becomes cheap and the lock is released quickly.

I added a test case that fails without the proposed fix.

---

• Tests are included.
• (Documentation change and updates are provided for the Apache Jena website)
• Commits have been squashed to remove intermediate development commit messages.
• Key commit messages start with the issue number (GH-xxxx)

By submitting this pull request, I acknowledge that I am making a contribution to the Apache Software Foundation under the terms and conditions of the Contributor's Agreement.

---

See the Apache Jena "Contributing" guide.

[Aklakan]

Intent: Avoid memory copy.

[rvesse]

Or maybe rowSet.stream().count() would be cleaner and still avoid the memory copy (though it has conversion to Stream overhead)?

[Aklakan]

good point! - changed to the cleaner stream version.

[afs]

Maybe it does not matter with this destructive operation but the cost of stream() is not trivial.
forEach does not use stream.

[afs]

Intent: Avoid memory copy.

RowSetMem rewind should not be a memory copy. It shares the rows array.

A rewindable RowSet (e.g. Mem) goes back to the beginning where as forEach continues the iteration.

[Aklakan]

Behavior so far (IIRC):

• For non-RowSetMem instances counting would create an intermediate in-memory copy.
• The count for a RowSetMem instances would be the total number of bindings in the in-memory row set - rather than the number of remaining bindings.

I added the method RowSetStream.forEachRemaining - the stack trace for RowSetOps.count using rowSet.stream().count() then becomes a delegate to the forEachRemaining method (with some stream creation overhead that should be neglectable for larger counts):

[Aklakan]

As a separate case where timeouts get ignored (besides the hash probe table construction), a cancel check is needed here for the following query on TDB2:

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?x { ?x rdfs:label ?y . ?y rdfs:label ?z }

time curl http://localhost:3030/ds --data-urlencode 'query=PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> SELECT ?x { ?x rdfs:label ?y . ?y rdfs:label ?z }' --data-urlencode 'timeout=1'

Against this preloaded TDB2 DBpedia dataset (~30GB)

[Aklakan]

Set to draft because I am still investigating this query on in memory datasets.

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?x { ?x rdfs:label ?y . ?y rdfs:label ?z }

[Aklakan]

So for the "empty join" case I had to add cancel signal checks to the StageMatchTriple (added a test case) and StageMatchTuple (so far only tested locally against a Fuseki). On a larger scale the issue is, that the iterator of graph.find() would actually already need cancellation support. For example, a custom graph.find() implementation could sequentially search a large file for matching triples - and if there are no matches in that file, then a call to hasNext would scan all data without any means to interrupt the process. Right?

[Aklakan]

To summarize:

• Aborting the now lazy HashJoin has a dedicated test.
• For the empty join case, there is a dedicated test for StageMatchTriple based on a custom GraphBase.find() implementation that returns an infinite stream of matches.
• But: not sure how I can mock a similar test for StageMatchTuple in TDB1/2, because it requires some large dataset or mocking NodeIds. Here is a brief screen-capture of the effect of this PR's code:

CancelStageMatchTuple.webm