unsafe mode

[missinglink]

this draft PR addresses some of the concerns in #203
it still requires: user testing, feedback and additional nodejs tests.

unsafe mode can be enabled by setting { unsafe: true } when creating the db connection.

this PR can be tested by adding a postinstall script to your package.json

"scripts": {
  "postinstall": "npm install --build-from-source missinglink/better-sqlite3#unsafe_mode",

[missinglink]

Is this something you would consider merging?

[JoshuaWise]

I think this is good solution for those who need the functionality. Thanks for contributing. I'll accept it when there are tests and documentation.

[missinglink]

testers!
this PR can be tested by adding a postinstall script to your package.json

"scripts": {
  "postinstall": "npm install --build-from-source missinglink/better-sqlite3#unsafe_mode",

[Prinzhorn]

I think it makes sense that unsafe also affects the case above, not just the iterators case. E.g. to allow things such as #338

And what about locked? Which use-case does that prevent and could this be disabled in unsafe mode as well?

[missinglink]

Please provide a test case which demonstrates the functionality and I'll include it.

[Prinzhorn]

@JoshuaWise could you elaborate what use-case stmt->locked prevents? I couldn't find tests (only for busy and iterators), but maybe I'm blind. I assume this would prevent executing the same statement inside itself? E.g. if you use a user-defined function inside a statement and the function implementation executes the same statement again? Wouldn't that end up in an endless recursion anyway? What I'm asking is: can you give an example code that makes better-sqlite3 go pew pew "This statement is busy executing a query"?

[missinglink]

I've added a test for my specific use-case in 7ad0b19

I would not suggest to anyone that they write to the same table they are iterating over unless they are 100% sure of what they are doing.

I'm personally willing to do this because my database is 40GB+ and I don't want to copy it.

In most cases it is preferable to use safe mode and simply copy the database, read from one and insert into the other.

Please be very careful when using unsafe mode!

[JoshuaWise]

@JoshuaWise could you elaborate what use-case stmt->locked prevents? I couldn't find tests (only for busy and iterators), but maybe I'm blind. I assume this would prevent executing the same statement inside itself? E.g. if you use a user-defined function inside a statement and the function implementation executes the same statement again? Wouldn't that end up in an endless recursion anyway? What I'm asking is: can you give an example code that makes better-sqlite3 go pew pew "This statement is busy executing a query"?

It protects against the example you gave, but it also protects against iterating a statement while already iterating that statement.

I'll have to look over all this discussion when I have more time. Hopefully I won't keep you waiting too long!

[JoshuaWise]

Unsafe mode is now available in v7.0.0 (although the API and implementation is a bit different from this PR). Read the docs to learn the details.

[chanibal]

@jeromew, @missinglink thank you, this is the 5th SQLite library I have tried to use in the last few days (3 on deno, 2 on nodejs - most dying on a mere 8GB file or this issue).
This is also the first one to work correctly with the use case from #203.
This unsafe mode is really needed.

Did you consider adding a note to the "TypeError: This database connection is busy executing a query" that unsafe mode will (dangerously) fix the issue? Might help with some frustration.