Making the Whole Project/Repo Menu-Driven

[avinashkranjan]

[UPDATE]: Check this comment for progress on this issue: #831 (comment)

Is your feature request related to a problem? Please describe.
Changing directories looking for scripts to use one by one will be quite a headache.

Describe the solution you'd like
Making it a menu-driven script, where a user can select the option and the specific script will run (can take arguments too if needed at runtime) in sort like Lazy Script if you are aware of.

Let me brief you about what you have to do on this issue:

1. Make a menu-driven script which will perform the task as per selected option.
2. There are 100+ scripts currently so we need to provide all in the menu.
   Which can be done in the following ways
   • Scripts can be further categorized into sub categories
     • Automation
     • AI/ML
     • Calculators
     • Games
     • Fun
     • Scrappers
     • Bots
       and More
   • It'll help in searching through the numerous script easy
3. Give all the folder 777 permission for the execution of python files
4. Do provide comments while doing scripting
5. Be aware of file path as per linux system ie. /root/your_filename while working on github
6. Follow all the guidelines provided

Marking Guidelines

• Since This is a big Issues, Multiple contributors are welcome to contribute to the script, Depending on the Contribution PR level would be marked..the Issue would be a Level3, Each Linked PR would be of Minimum Level2.

@kaustubhgupta @antrikshmisri @santushtisharma10 Kindly Helpout the Contributors working on this particular Issue..As this one is a complex issue..If Any Issues, DM me on Discord..

Happy Coding..👨🏻‍💻

[AshuKV]

Can I work on this issue under GSSOC21? Kindly assign it to me.

[XZANATOL]

I've a suggestion. We can start by categorizing each script into a dictionary.. something like:
{"Scrappers": ["Project1", "Project2"], "Automation":["Project3", "Project4"]}
Each list can contain records of scripts that are in another category. This will assure that effiency of sorting and tag searching, and put each script into its suitable place making it organized. This dictionary can be saved in a CSV file for future adds/edits.

We can also make a function to act as an API which adds a new script with the suitable tags, and help contributors to automate the process of adding their scripts to this script instead of manually adding them. (Let's leave that for after we settle on how we will save the data of the existing scripts.)

[Ajay-Singh-Rana]

Hey..! What if we create a script that takes folder names as arguments and lets the user decide which folder thier script is to be moved in to and also an option to add a newer category.
script.py folder_name category
or
script.py -n new_category folder_name [folder_name [folder_name]

[kaustubhgupta]

Hi, @Ajay-Singh-Rana @XZANATOL @AshuKV,

As I have just returned home, I need some time to plan out the overall structure of the script, how it will handle new scripts and how we can have a GUI for that.

I will update you on this day after tomorrow

[Ajay-Singh-Rana]

Any updates..?

[kaustubhgupta]

Any updates..?

Will update by EOD

[avinashkranjan]

Hey @kaustubhgupta If Need any help or any doubt kindly text me on discord..Participants are asking about this Issue..

[kaustubhgupta]

Hi, @AshuKV @XZANATOL @avinashkranjan @Ajay-Singh-Rana (Apologies for updating late, was busy with important work),

I liked the @XZANATOL implementation to create the database of scripts based on categories. I think a big JSON would do the work. Why JSON? Because it can be loaded into the dictionary in Python. Creating this JSON would be a manual task initially as we can't trace back to every merged pull request and check if it was a scrapper or ML script as neither the participants have filled the PR template properly nor there was an option to add a category. @Ajay-Singh-Rana Your idea has a problem that a new user would not be aware of the folder where the script lies.

Also, one major problem apart from these ideas is that all participants have different entry points for their script. Some of them have main.py, scrapper.py, or script-specific names.

Here is my workaround for this issue in 4 tasks (Please read till the end before assuming why I am asking these):

• Creating the manual JSON: This JSON file, as suggested by @XZANATOL will have the root element as Category and its value as another JSON with the following information:

  • Script Name
  • Script folder relative path
  • Entry point of the script (Which file runs the program)
  • arguments it takes (if any). Now here we need to define that if the arguments will be passed using argument parser or direct CLI.
  • requirements_path: None if no requirements.txt file present else the relative path of this file
  • Contributor Name
  • Small Description

• Creating the demo Script for testing: After this database is prepared, we can start working on the implementation part. This demo script will present the user with categories of the script. Then as per user input, a list of available scripts will be presented. and then the user input script will be run. The script can be run by the subprocess module. Why not OS? Because in the subprocess module we can define the script entry point and arguments more easily. Refer this article for implementation. Now as soon as the user gives the script choice, our script should use the database to see if it has requirements.txt. If yes then first run: pip install -r $requirements_path/requirements.txt and then find the entry point of the script to call: python entrypoint.py. This command can be called via subprocess too with arguments.

• Porting this demo to GUI: If this demo script works, then we can port this to a suitable GUI with a menu to select the category, sub-script, and provide arguments if needed. We can also have a checkbox for installing requirements.txt! The GUI can be made in any framework depending upon which contributor takes this sub-issue.

• Automating JSON updation: The manual information adding is a one-time task. To add new script information to this JSON, we will use the PR template and GitHub Actions to add update this information. The PR template needs to be updated with the new parameters mentioned in the manual JSON work and every participant will have to compulsorily fill in this information. The GitHub action, on every PR merged, will push this information to this JSON and hence an application will be converted out of this repo! When this task will be performed. all other PRs will be stopped for a while and they will need to update the PR template accordingly.

• BONUS task, website updation: The website currently has hardcoded information about the scripts. The JSON generated can be used by web designers to display script information with contributor name, small description, and link to the script. That's why I asked in the PR template this information in the first place! This will help both projects!

As these tasks are themselves way too big, every task will be rewarded level3. Also, no separate issue will be created for this. All 4-5 PRs will be merged tagging this issue only. The subtasks will be assigned and ticked off in the order mentioned below in this comment only:

• Creating Manual JSON - @XZANATOL (Added 'Master Script' base JSON DB #963)
• Creating demo Script - @devRawnie
• Automating JSON updation - @XZANATOL (Action db update #1025)
• BONUS task, Website updation - @Ayushjain2205 (Updating Scripts contributors on Website with implementation of search bar #1030)
• Porting demo to GUI

Let me know what should be modified, any new ideas except this, or anything else. Also, which task you are interested in.

VERY IMPORTANT INFO: THIS WHOLE ISSUE IS NOT FIRST COME FIRST SERVE. IF YOU CAN GENUINELY HELP AND HAVE THE SKILLS TO CODE THIS THEN ONLY COMMENT AND DISCUSS.

I have created a separate discussion thread for this. Join here: #882