You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Make program work with depth > 1. Fix parsing of urls
Dump data regularly to file to avoid too large a tree
Incorrect Settings.json should not crash the program, issue an error message.
Add a quit command
Lacking a Settings.json, the user should be given the option to generate a default one.
Add ability to read robots.txt and avoid sites that should not be accessed (patterns like "?id=")
Make robot more polite (make sure not to flood a site, and make it a default but optional setting)
Add ability to deal with ssl or handshakes. (Curl does this)
More customizability, make settings a polymorphic class that has functions to alter logic? (<< this part will be implemented with a scripting language)
Better linux console.
Sort the libraries, they should be downloadable, rather than embedded already.
Make program more into a state machine, polymorphic
The text was updated successfully, but these errors were encountered:
Implement python as the scripting backend, and have it able to sort and manipulate data
(Two examples i can think of are: Have all of the text be in a set; Search for the tags named "object" and "price" under the same dom and output it as a sortable column)
Make a solution work as a dynamic/static library
Write documentation/wiki and usage for the said library
TODO:
The text was updated successfully, but these errors were encountered: