flatpak support?

Propose changes for dramatically faster scanning and fallback options

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,26 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: "[BUG]"
+labels: bug
+assignees: Novattz
+
+---
+
+**Describe the bug**
+- A clear and concise description of what the bug is.
+
+**Terminal output** 
+- Copy and paste the entire terminal output when running the script.
+
+**Log File Output**
+- If the script logged any errors, attach the log file **`script.log`** to this issue.
+
+**Debug Output**
+- Run the script with the **`--debug`** argument and attach the log file **`debug_script.log`** to this issue.
+
+**Steam library path**
+- Provide the path where your steam library is.
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,20 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: "[ Feature Request ]"
+labels: enhancement
+assignees: Novattz
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

README.md

@@ -1,35 +1,38 @@
-# Steam DLC Fetcher and installer for Linux
-- Python script designed for linux to automate fetching of DLC id's for steam games and the installation of creamlinux automatically.
 
-# Features
+# Steam DLC Fetcher and installer for Linux
+- Python script designed for linux to automate fetching of DLC id's for steam games and the installation of creamlinux automatically. [Demo](https://www.youtube.com/watch?v=22LDDUoBvus&ab_channel=Nova)
+### Features
 - Automatically fetches and lists DLC's for selected steam game(s) installed on the computer.
 - Automatically installs creamlinux and its components into selected steam games, excluding and handling specific config files.
 - Provides a simple cli to navigate and operate the entire process.
 
-# Demo
-https://www.youtube.com/watch?v=22LDDUoBvus&ab_channel=Nova
-
-# To do
-- Cross reference dlc files and dlc id's. Incase dlc id's and dlc files differ in terms of quantity it will notify the user.
-- Possibly add functionality to search for dlc files/automatically installing them.
-- Add the possibility to install cream/smokeapi for games running proton.
-- Check if the game already has dlc files installed
-- Gui?
-
-# Prerequisites
+## Usage
+### Prerequisites
 - `python 3.x`
 - `requests` library
 - `zipfile` library
 
-# Usage
+
+### Installation
+
 - Clone the repo or download the script.
 - Navigate to the directory containing the script.
 - Run the script using python.
-```bash
-python dlc_fetcher.py
-```
 
-# Issues?
+#### OR
+Use this one-line shell script.
+```bash
+git clone https://github.com/Novattz/creamlinux-installer;cd creamlinux-installer;python dlc_fetcher.py
+```
+## TODO
+- [ ] Cross reference dlc files and dlc id's. Incase dlc id's and dlc files differ in terms of quantity it will notify the user.
+- [ ] Possibly add functionality to search for dlc files/automatically installing them.
+- [ ] Add the possibility to install cream/smokeapi for games running proton.
+- [ ] Check if the game already has dlc files installed
+- [ ] Gui?
+- [ ] Add checker for configs already applied to games. (i.e script will check for new dlc id's for already applied games.)
+
+### Issues?
 - Open a issue and attach all relevant errors/logs.
 
 # Credits

dlc_fetcher.py

@@ -4,24 +4,58 @@ import requests
 import zipfile
 import time
 import stat
+import subprocess
+import psutil
+from collections import defaultdict
+import logging
+import argparse
 
 LOG_FILE = 'script.log'
+DEBUG_FILE = 'debug_script.log'
+TIMEOUT = 180  # Timeout in seconds (3 minutes)
+
+def setup_logging(debug):
+    log_format = '%(asctime)s [%(levelname)s] %(message)s'
+    date_format = '%m-%d %H:%M:%S'
+    if debug:
+        logging.basicConfig(filename=DEBUG_FILE, level=logging.DEBUG, format=log_format, datefmt=date_format)
+    else:
+        logging.basicConfig(filename=LOG_FILE, level=logging.ERROR, format=log_format, datefmt=date_format)
 
 def clear_screen():
     os.system('cls' if os.name == 'nt' else 'clear')
 
+def log_error(message):
+    logging.error(message)
+    print(message)
+
+def log_debug(message):
+    logging.debug(message)
+
+def read_steam_registry():
+    registry_path = os.path.expanduser('~/.steam/registry.vdf')
+    if os.path.exists(registry_path):
+        log_debug(f"Found Steam registry file: {registry_path}")
+        with open(registry_path, 'r') as f:
+            content = f.read()
+            install_path = re.search(r'"InstallPath"\s*"([^"]+)"', content)
+            if install_path:
+                return install_path.group(1)
+    return None
+
 def fetch_latest_version():
     try:
         response = requests.get("https://api.github.com/repos/Novattz/creamlinux-installer/releases/latest")
         data = response.json()
         return data['tag_name']
     except requests.exceptions.RequestException as e:
-        log_message(f"Failed to fetch latest version: {str(e)}")
+        log_error(f"Failed to fetch latest version: {str(e)}")
         return "Unknown"
 
-def show_header(app_version):
+def show_header(app_version, debug_mode):
     clear_screen()
     cyan = '\033[96m'
+    red = '\033[91m'
     reset = '\033[0m'
     print(f"{cyan}")
     print(r"""
@@ -39,13 +73,12 @@ def show_header(app_version):
     > Version: {app_version}
     {reset}
     """)
+    if debug_mode:
+        print(f"{red}    [Running in DEBUG mode]{reset}\n")
+    print()
 
 app_version = fetch_latest_version()
-
-def log_message(message):
-    with open(LOG_FILE, 'a') as log_file:
-        log_file.write(f"{message}\n")
-    print(message)
+#app_version = "TESTING / LETS NOT OVERLOAD GITHUB FOR NO REASON"
 
 def parse_vdf(file_path):
     library_paths = []
@@ -55,48 +88,104 @@ def parse_vdf(file_path):
             paths = re.findall(r'"path"\s*"(.*?)"', content, re.IGNORECASE)
             library_paths.extend([os.path.normpath(path) for path in paths])
     except Exception as e:
-        log_message(f"Failed to read {file_path}: {str(e)}")
+        log_error(f"Failed to read {file_path}: {str(e)}")
     return library_paths
 
-def find_steam_library_folders():
-    base_paths = [
+def find_steam_binary():
+    try:
+        result = subprocess.run(['which', 'steam'], stdout=subprocess.PIPE)
+        steam_path = result.stdout.decode('utf-8').strip()
+        if steam_path:
+            return os.path.dirname(steam_path)
+    except Exception as e:
+        log_error(f"Failed to locate steam binary: {str(e)}")
+    return None
+
+def find_steam_library_folders(manual_path=""):
+    search_list = [
+        # Default
         os.path.expanduser('~/.steam/steam'),
         os.path.expanduser('~/.local/share/Steam'),
-        os.path.expanduser('~/home/deck/.steam/steam'),
-        os.path.expanduser('~/home/deck/.local/share/Steam'),
-        '/mnt', '/media',
-        '/run/media/mmcblk0p1/steamapps'
+
+        # Steam Deck
+        os.path.expanduser('/home/deck/.steam/steam'),
+        os.path.expanduser('/home/deck/.local/share/Steam'),
+
+        # Others
+        '/mnt/Jogos/Steam',
+        '/run/media/mmcblk0p1',
+
+        # Flatpak
+        os.path.expanduser('~/.var/app/com.valvesoftware.Steam/.local/share/Steam'),
+        os.path.expanduser('~/.var/app/com.valvesoftware.Steam/data/Steam/steamapps/common')
     ]
+
     library_folders = []
     try:
-        for base_path in base_paths:
-            if os.path.exists(base_path):
-                for root, dirs, files in os.walk(base_path, topdown=True):
-                    if 'steamapps' in dirs:
-                        steamapps_path = os.path.join(root, 'steamapps')
+        if manual_path != "" and manual_path != None:
+            log_debug(f"Manual game path set to \"{manual_path}\" skipping path lookup")
+            library_folders.append(manual_path)
+
+        else:
+            steam_binary_path = find_steam_binary()
+            if steam_binary_path and steam_binary_path not in search_list:
+                log_debug(f"Found Steam Binary path! adding it to search paths: {steam_binary_path}")
+                search_list.append(steam_binary_path)
+
+            steam_install_path = read_steam_registry()
+            if steam_install_path and steam_install_path not in search_list:
+                log_debug(f"Found Steam Binary path! adding it to search paths: {steam_install_path}")
+                search_list.append(steam_install_path)
+
+            log_debug(f"Paths that will be searched: {search_list}")
+
+            for search_path in search_list:
+                if os.path.exists(search_path):
+                    log_debug(f"Scanning path: {search_path}")
+
+                    steamapps_path = str(os.path.normpath(f"{search_path}/steamapps"))
+                    if os.path.exists(steamapps_path):
                         library_folders.append(steamapps_path)
+                        log_debug(f"Found steamapps folder: {steamapps_path}")
+
                         vdf_path = os.path.join(steamapps_path, 'libraryfolders.vdf')
                         if os.path.exists(vdf_path):
+                            log_debug(f"Found libraryfolders.vdf: {vdf_path}")
                             additional_paths = parse_vdf(vdf_path)
                             for path in additional_paths:
                                 new_steamapps_path = os.path.join(path, 'steamapps')
                                 if os.path.exists(new_steamapps_path):
                                     library_folders.append(new_steamapps_path)
-                            dirs[:] = []  # Prevent further scanning into subdirectories
-        if not library_folders:
-            raise FileNotFoundError("No Steam library folders found.")
+                                    log_debug(f"Added additional steamapps folder: {new_steamapps_path}")
+
+            if not library_folders:
+                raise FileNotFoundError("No Steam library folders found.")
+            log_debug(f"Total Steam library folders found: {len(library_folders)}")
     except Exception as e:
-        log_message(f"Error finding Steam library folders: {e}")
+        log_error(f"Error finding Steam library folders: {e}")
+        log_error("Scanned paths:")
+        for path in search_list:
+            log_error(f"  - {path}")
     return library_folders
 
 def find_steam_apps(library_folders):
     acf_pattern = re.compile(r'^appmanifest_(\d+)\.acf$')
     games = {}
+    scanned_folders = []
+    start_time = time.time()
+
     try:
         for folder in library_folders:
+            if time.time() - start_time > TIMEOUT:
+                log_error("Script timeout reached. Stopping Steam app scan.")
+                return games
+            scanned_folders.append(folder)
             if os.path.exists(folder):
+                log_debug(f"Scanning folder for ACF files: {folder}")
+                acf_count = 0
                 for item in os.listdir(folder):
                     if acf_pattern.match(item):
+                        acf_count += 1
                         try:
                             app_id, game_name, install_dir = parse_acf(os.path.join(folder, item))
                             if app_id and game_name:
@@ -104,12 +193,20 @@ def find_steam_apps(library_folders):
                                 if os.path.exists(install_path):
                                     cream_installed = 'Cream installed' if 'cream.sh' in os.listdir(install_path) else ''
                                     games[app_id] = (game_name, cream_installed, install_path)
+                                    log_debug(f"Found game: {game_name} (App ID: {app_id})")
                         except Exception as e:
-                            log_message(f"Error parsing {item}: {e}")
+                            log_error(f"Error parsing {item}: {e}")
+                log_debug(f"Found {acf_count} ACF files in {folder}")
         if not games:
             raise FileNotFoundError("No Steam games found.")
+        log_debug(f"Total games found: {len(games)}")
     except Exception as e:
-        log_message(f"Error finding Steam apps: {e}")
+        log_error(f"Error finding Steam apps: {e}")
+
+        log_error("Scanned folders:")
+        for folder in scanned_folders:
+            log_error(f"  - {folder}")
+
     return games
 
 def parse_acf(file_path):
@@ -121,7 +218,7 @@ def parse_acf(file_path):
         install_dir = re.search(r'"installdir"\s+"([^"]+)"', data)
         return app_id.group(1), name.group(1), install_dir.group(1)
     except Exception as e:
-        log_message(f"Error reading ACF file {file_path}: {e}")
+        log_error(f"Error reading ACF file {file_path}: {e}")
         return None, None, None
 
 def fetch_dlc_details(app_id):
@@ -130,7 +227,7 @@ def fetch_dlc_details(app_id):
         response = requests.get(base_url)
         data = response.json()
         if app_id not in data or "data" not in data[app_id]:
-            log_message("Error: Unable to fetch game details.")
+            log_error("Error: Unable to fetch game details.")
             return []
         game_data = data[app_id]["data"]
         dlcs = game_data.get("dlc", [])
@@ -146,17 +243,17 @@ def fetch_dlc_details(app_id):
                         dlc_name = dlc_data[str(dlc_id)]["data"].get("name", "Unknown DLC")
                         dlc_details.append({"appid": dlc_id, "name": dlc_name})
                     else:
-                        log_message(f"Data missing for DLC {dlc_id}")
+                        log_error(f"Data missing for DLC {dlc_id}")
                 elif dlc_response.status_code == 429:
-                    log_message("Rate limited! Please wait before trying again.")
+                    log_error("Rate limited! Please wait before trying again.")
                     time.sleep(10)
                 else:
-                    log_message(f"Failed to fetch details for DLC {dlc_id}, Status Code: {dlc_response.status_code}")
+                    log_error(f"Failed to fetch details for DLC {dlc_id}, Status Code: {dlc_response.status_code}")
             except Exception as e:
-                log_message(f"Exception for DLC {dlc_id}: {str(e)}")
+                log_error(f"Exception for DLC {dlc_id}: {str(e)}")
         return dlc_details
     except requests.exceptions.RequestException as e:
-        log_message(f"Failed to fetch DLC details for {app_id}: {e}")
+        log_error(f"Failed to fetch DLC details for {app_id}: {e}")
         return []
 
 def install_files(app_id, game_install_dir, dlcs, game_name):
@@ -181,14 +278,32 @@ def install_files(app_id, game_install_dir, dlcs, game_name):
             print(f"Custom cream_api.ini has been written to {game_install_dir}.")
             print(f"Installation complete. Set launch options in Steam: 'sh ./cream.sh %command%' for {game_name}.")
         else:
-            log_message("Failed to download the files needed for installation.")
+            log_error("Failed to download the files needed for installation.")
     except Exception as e:
-        log_message(f"Failed to install files for {game_name}: {e}")
+        log_error(f"Failed to install files for {game_name}: {e}")
 
 def main():
-    show_header(app_version)
+    parser = argparse.ArgumentParser(description="Steam DLC Fetcher")
+    parser.add_argument("--manual", metavar='steamapps_path', help="Sets the steamapps path for faster operation", required=False)
+    parser.add_argument("--debug", action="store_true", help="Enable debug logging")
+    args = parser.parse_args()
+
+    setup_logging(args.debug)
+    show_header(app_version, args.debug)
+
     try:
-        library_folders = find_steam_library_folders()
+        library_folders = find_steam_library_folders(args.manual)
+        if library_folders == [] or library_folders == None:
+            print("Falling back to Manual Method since no library folder was found")
+            steamapps_path = input("Steamapps Path: ")
+
+            if len(steamapps_path) > 3:
+                library_folders = [ steamapps_path ]
+
+            else:
+                print("Invalid path! Closing the program...")
+                return
+
         games = find_steam_apps(library_folders)
         if games:
             print("Select the game for which you want to fetch DLCs:")
@@ -216,7 +331,8 @@ def main():
         else:
             print("No Steam games found on this computer or connected drives.")
     except Exception as e:
-        log_message(f"An error occurred: {e}")
+        logging.exception("An error occurred:")
+        log_error(f"An error occurred: {e}")
 
 if __name__ == "__main__":
     main()