Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Todo #1

Open
kingRodian opened this issue Dec 5, 2016 · 2 comments
Open

Todo #1

kingRodian opened this issue Dec 5, 2016 · 2 comments

Comments

@kingRodian
Copy link
Contributor

kingRodian commented Dec 5, 2016

TODO:

    • Make program work with depth > 1. Fix parsing of urls
    • Dump data regularly to file to avoid too large a tree
    • Incorrect Settings.json should not crash the program, issue an error message.
    • Add a quit command
    • Lacking a Settings.json, the user should be given the option to generate a default one.
    • Add ability to read robots.txt and avoid sites that should not be accessed (patterns like "?id=")
    • Make robot more polite (make sure not to flood a site, and make it a default but optional setting)
    • Add ability to deal with ssl or handshakes. (Curl does this)
    • More customizability, make settings a polymorphic class that has functions to alter logic? (<< this part will be implemented with a scripting language)
    • Better linux console.
    • Sort the libraries, they should be downloadable, rather than embedded already.
    • Make program more into a state machine, polymorphic
OriginLive pushed a commit that referenced this issue Dec 12, 2016
@OriginLive
Copy link
Owner

  • Tron's suggestion, make console polymorphic, use string view for text (fix the concatenation)

@OriginLive
Copy link
Owner

OriginLive commented Mar 2, 2017

  • Implement python as the scripting backend, and have it able to sort and manipulate data
    (Two examples i can think of are: Have all of the text be in a set; Search for the tags named "object" and "price" under the same dom and output it as a sortable column)
  • Make a solution work as a dynamic/static library
  • Write documentation/wiki and usage for the said library
  • Multithreading
  • Object / lambda commands as opposed to void*
  • Polish

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants