From Newsgroup: alt.comp.os.windows-11
Lawrence DoOliveiro wrote:
On Sun, 22 Nov 2026 02:17:32 -0400, Maria Sophia wrote:
What's different between the batch and python is mostly:
a. SQLite queries primarily use Python's built-in sqlite3 module.
I like this alternative <https://rogerbinns.github.io/apsw> to the standard-library sqlite module.
Pros and cons: <https://rogerbinns.github.io/apsw/pysqlite.html>.
When doing lots of database queries, I find a function like the
following very useful for handling the repetitive chores of 1)
creating a cursor, 2) executing a query on the cursor, 3) fetching the results, and 4) disposing of the cursor. APSW version:
def db_iter(conn, cmd, values = None, mapfn = lambda x : x) :
"executes cmd on a new cursor from connection conn and yields" \
" the results in turn."
for item in conn.cursor().execute(cmd, values) :
yield mapfn(item)
#end for
#end db_iter
Standard-library sqlite version:
def db_iter(conn, cmd, values = (), mapfn = lambda x : x) :
"executes cmd on a new cursor from connection conn and yields" \
" the results in turn."
for item in conn.cursor().execute(cmd, values) :
yield mapfn(item)
#end for
#end db_iter
As a result of the rewrite, I'm beginning to less-hate python since
it's actually more intuitive than the Windows batch scripting was. :)
Surely *anything* would be more intuitive than Windows batch scripting
Hi Lawrence,
There's a lot to unpack there on the "intuition", as you may not know I learned how to code in the 1970's on Fortran IV, IBM Assembly, COBOL, PL/I, etc., where, over time, I evolved to hate coding with a passion because
every damn language does the same exact thing (more or less), in the end,
but with frustratingly irrationally different syntax requirements.
It's kind of like religions. There are so many to choose from, and yet
they're mostly mutually exclusive, and none of them are based on reality.
There's merit in picking the one with the loosest set of rules that does
the job, e.g., pick a religion that allows vice and alcohol to coexist.
I pick the language with the least restrictive syntax, usually.
So I tend to "code" in shell scripts, simply because those are the commands
my fingers use when I need to get something done fast. Yet, the port to
Python needed vastly fewer crutches than the Windows shell script needed.
I appreciate your APSW (Another Python SQLite Wrapper) suggestion, and that you shared a generator helper (db_iter) to cut down on repetitive query boilerplate.
I didn't choose SQL, Nova did, but I can see that APSW might better mirror SQLite's API & behavior, which would be helpful when extending the script. However, it's an external package (pip install apsw) so that increases dependencies, although there might be useful tradeoffs in doing so.
But I see that instead of writing cur = conn.cursor(), cur.execute(...),
and cur.fetchall() in every query function, your suggestion allows us to create a cursor, run a cmd with bound values, and yield rows one at a time
as a generator instead of loading everything into a list up front, where
mapfn lets us optionally transform each row on the fly to pull a single column.
I found some minor glitches in the v1p9 code so I fixed them but they
were minor enough to not need to uprev the version from the last fix.
#!/usr/bin/env python3
# ----------------------------------------------------------------------
# adbfolder.py
# Usage: python adbfolder.py
# ----------------------------------------------------------------------
# Outputs a perfect textual inventory, with debug capability, of an Android
# homescreen, including location of all panes, folders, packages and titles.
#
# Requires adb, sqlite3, tabulate and Nova Launcher (on Android).
# Optionally requires aapt/aapt2 to determine labels on orphan packages.
#
# Intended to assist in debug, inventory, archival, backup and restore.
# Tested on Windows over Wi-Fi/USB, but should work on macOS/Linux also.
#
# Will pull a backup from android via adb or read an existing nova db or
# nova backup archive on windows needing only adb and sql dependencies
#
# List order is top to bottom, left to right, meaning the leftmost pane
# is 1st, then subsequent panes, from 0 to 1 to 2 and so on to the right.
#
# Similarly, the topleft folder is 1st, then the next folder to the right.
#
# Likewise, packages and titles inside of folders are listed in top to
# bottom order, left to right, providing an exact textual description.
#
# Bare icons, not inside folders, are listed separately as a line item.
# The dock and folders in the dock are treated much like the panes are.
#
# This script not only provides a textual inventory of the name and
# homescreen location of every folder name, package name and display
# name ever added to the homescreen, but it optionally will 'diff'
# against packages which were manually installed by the user but
# which have no current homescreen icon (i.e., "headless" apps).
#
# Note the display_name is the current icon title for a given app icon,
# which can be changed by the user, so it's not always the original.
# However, for apps installed by the user but without homescreen icons,
# the optional lookup pulls the app's original defined Android app_label.
#
# Output is appended to a timestamped running log file for traceability.
# Unless the user wipes out the running log, this becomes an automatic
# audit of all the physical changes a user has made to the homescreen.
# ----------------------------------------------------------------------
# Q: Why is saying YES to "Resolve app titles via adb/aapt" very slow?
# A: For each of the unplaced packages, the script does:
# a. adb shell dumpsys package <pkg> to look for nonLocalizedLabel=
# b. If that fails: adb shell pm path <pkg> to find the APK location
# c. adb pull <apk> to download the APK to the PC
# d. aapt dump badging <apk> to read the app label from the APK
# Which is up to 4 ADB commands times N apps = 4N round-trips.
# Over USB, each round-trip is fast (milliseconds).
# But, over Wi-Fi, each one has wireless handshake latency,
# so the total time adds up just waiting on ADB to return results.
# ----------------------------------------------------------------------
# v1p9 20261122 Replaced redundant which() wrapper with a direct
# shutil.which import so that which("adb"), which("sqlite3"),
# which("aapt") and which("aapt2") all work out of the box.
# v1p8 20261121 Fixed bugs in the optional search for orphan packages
# v1p7 20261121 Modified the closing comments to reflect the python flow
# Fixed passing of capture_output=True and text=True twice by
# mistake in the helper section.
# v1p6 20261121 Ported to adbfolder.py v1p6 (took a half-dozen iterations)
# Matched output of abfolder.bat v6p1 with adbfolder.py v1p6
# v1p0 20261121 First attempt at porting adbfolder.bat to adbfolder.py
# What's different between the batch and python is mostly:
# a. SQLite queries primarily use Python's built-in sqlite3 module.
# b. Output tables use the third-party tabulate module.
# c. Archive extraction uses Python's zipfile.
# d. Filename sanitization uses Python regular expressions.
# e. There are no generated PowerShell scripts or BAT subroutines.
# f. The optional missing-package check uses Python sets to
# compare installed packages against Nova database packages.
# g. Debugging is enabled with DEBUG = True or the ADBFOLDER_DEBUG=1
# environment variable.
# ----------------------------------------------------------------------
# ----------------------------------------------------------------------
import os
import sys
import re
import subprocess
import shutil
import sqlite3
import zipfile
import logging
from shutil import which
from pathlib import Path
from datetime import datetime
from tabulate import tabulate
# ------------------------------------------------------------------------
# Configuration
# ------------------------------------------------------------------------
DEBUG = False # set True for verbose traces
ADB_EXE = Path(r"C:\app\editor\android\scrcpy\adb.exe")
SQLITE_EXE = Path(r"C:\app\editor\android\sdk\platform-tools\sqlite3.exe")
WORK_DIR = Path(r"C:\app\editor\android\scrcpy\nova")
ANDROID_BCK_DIR = "/storage/emulated/0/0000/bck/nova_backup"
DEFAULT_CONTAINER = 2921
TEMP = Path(os.environ.get("TEMP", Path.home() / "tmp"))
def debug(msg):
if DEBUG:
print(f"[DEBUG] {msg}")
# ------------------------------------------------------------------------
# Helpers
# ------------------------------------------------------------------------
def run(cmd, **kwargs):
"""Run a command and return CompletedProcess."""
debug(f"Running: {' '.join(map(str, cmd))}")
kwargs.setdefault("capture_output", True)
kwargs.setdefault("text", True)
kwargs.setdefault("encoding", "utf-8")
kwargs.setdefault("errors", "replace")
kwargs.setdefault("timeout", 60)
return subprocess.run(cmd, **kwargs)
def ensure_dir(path: Path):
path.mkdir(parents=True, exist_ok=True)
def prompt(msg, default=None):
if default is not None:
msg = f"{msg} [Default: {default}]: "
else:
msg = f"{msg}: "
val = input(msg).strip()
return val if val else (default or "")
def yes_no(msg, default="n"):
val = prompt(f"{msg} [y/N]" if default != "y" else f"{msg} [Y/n]", default)
return val.lower().startswith("y")
# ------------------------------------------------------------------------
# Dependency check
# ------------------------------------------------------------------------
def check_dependencies():
print("Checking required tools (adb and sqlite3) ...")
# ADB
global ADB_EXE
path_adb = which("adb")
if not ADB_EXE.exists():
if path_adb:
print(f"[INFO] Configured ADB_EXE not found; using PATH: {path_adb}")
ADB_EXE = Path(path_adb)
else:
print(f"[WARN] adb.exe not found at '{ADB_EXE}' and not in PATH.")
else:
if not path_adb:
print(f"[INFO] adb.exe found at configured path, though not in PATH.")
# SQLite
global SQLITE_EXE
path_sqlite = which("sqlite3")
if not SQLITE_EXE.exists():
if path_sqlite:
print(f"[INFO] Configured SQLITE_EXE not found; using PATH: {path_sqlite}")
SQLITE_EXE = Path(path_sqlite)
else:
print(f"[ERROR] sqlite3.exe not found at '{SQLITE_EXE}' and not in PATH.")
input("Press Enter to exit...")
sys.exit(1)
else:
if not path_sqlite:
print(f"[INFO] sqlite3.exe found at configured path, though not in PATH.")
ensure_dir(WORK_DIR)
print(f"Working directory: {Path.cwd()}")
print(f"Nova work directory: {WORK_DIR}")
print("----------------")
print()
# ------------------------------------------------------------------------
# ADB helpers
# ------------------------------------------------------------------------
def adb_devices():
"""Return list of connected device serials."""
res = run([str(ADB_EXE), "devices"])
devices = []
for line in res.stdout.strip().splitlines()[1:]:
parts = line.split()
if parts and "\t" in line or len(parts) >= 2:
serial = parts[0]
state = parts[1] if len(parts) > 1 else ""
if state == "device":
devices.append(serial)
return devices
def adb_shell(serial, cmd):
return run([str(ADB_EXE), "-s", serial, "shell", cmd])
def adb_pull(serial, remote, local):
return run([str(ADB_EXE), "-s", serial, "pull", remote, str(local)])
def adb_push(serial, local, remote):
return run([str(ADB_EXE), "-s", serial, "push", str(local), remote])
# ------------------------------------------------------------------------
# Backup pull from Android
# ------------------------------------------------------------------------
def pull_from_android():
print("--- Pull Nova Backup from Android ---")
if not ADB_EXE.exists():
print(f"[ERROR] adb.exe not found at: '{ADB_EXE}'")
input("Press Enter to exit...")
sys.exit(1)
devices = adb_devices()
if not devices:
print("[ERROR] No connected ADB devices found.")
input("Press Enter to exit...")
sys.exit(1)
serial = devices[0]
debug(f"ADB_TARGET={serial}")
print(f"Targeting device: {serial}")
make_bck = yes_no("Open Nova Launcher settings on device to create a new backup first?")
if make_bck:
print("Launching Nova Launcher settings on device ...")
print("Tap Backup & restore > Backup > Save to save to your device")
run([
str(ADB_EXE), "-s", serial, "shell",
"am", "start",
"-n",
"com.teslacoilsw.launcher/com.teslacoilsw.launcher.preferences.SettingsActivity"
])
input("Create your backup in Nova Launcher on the device, then press ENTER to list backups.")
print(f"Listing backups in {ANDROID_BCK_DIR} ...")
print()
# List backups in default dir
res = adb_shell(serial, f"ls -1t '{ANDROID_BCK_DIR}'/*.novabackup 2>/dev/null")
backups = parse_adb_lines(res.stdout)
if not backups:
print(f"[INFO] No .novabackup files found in {ANDROID_BCK_DIR}.")
print("Searching Android storage for *.novabackup files ...")
print()
search_cmd = (
"for d in /storage/emulated/0 /storage/*; "
"do [ -d \"$d\" ] && [ \"$d\" != \"/storage/emulated\" ] && "
"find \"$d\" -maxdepth 5 -name '*.novabackup' 2>/dev/null; done"
)
res = adb_shell(serial, search_cmd)
backups = parse_adb_lines(res.stdout)
if not backups:
print(f"[WARN] No .novabackup files found in {ANDROID_BCK_DIR} or Android storage.")
remote_bck = manual_backup_path()
else:
print()
for i, (fname, fpath) in enumerate(backups, 1):
print(f" [{i}] {fname}")
pick = prompt(f"Select backup number [1-{len(backups)}, Default: 1]", "1")
try:
idx = int(pick) - 1
if not (0 <= idx < len(backups)):
raise ValueError
except ValueError:
# Treat as manual path
remote_bck = manual_backup_path(pick)
else:
fname, fpath = backups[idx]
remote_bck = fpath
return download_and_extract(serial, remote_bck)
def parse_adb_lines(text):
"""Parse ADB output into list of (filename, fullpath)."""
results = []
for line in text.splitlines():
line = line.strip().replace("\r", "")
if not line:
continue
path = Path(line)
fname = path.name
results.append((fname, line))
# Newest first, matching `ls -1t`
results.sort(key=lambda x: x[0], reverse=True)
return results
def manual_backup_path(prefill=""):
if prefill:
bck_file = prefill.strip().strip('"')
else:
bck_file = prompt(
"Enter backup filename or full Android path "
"e.g. 2026-11-19_12-57.novabackup"
)
if not bck_file:
print("[ERROR] No backup filename provided.")
input("Press Enter to exit...")
sys.exit(1)
remote_bck = bck_file
if not remote_bck.startswith("/"):
remote_bck = f"{ANDROID_BCK_DIR}/{remote_bck}"
fname = Path(remote_bck).name
return remote_bck
def download_and_extract(serial, remote_bck):
local_bck = WORK_DIR / Path(remote_bck).name
local_zip = WORK_DIR / "backup.zip"
extract_dir = WORK_DIR / "backup_extracted"
print()
print(f'Pulling "{remote_bck}" to "{local_bck}" ...')
res = adb_pull(serial, remote_bck, local_bck)
if res.returncode != 0:
print("[ERROR] Failed to pull backup from Android.")
input("Press Enter to exit...")
sys.exit(1)
if local_zip.exists():
local_zip.unlink()
# Rename pulled file to backup.zip
shutil.move(str(local_bck), str(local_zip))
if extract_dir.exists():
shutil.rmtree(extract_dir)
print(f"Extracting backup.zip to '{extract_dir}' ...")
try:
with zipfile.ZipFile(local_zip, "r") as zf:
zf.extractall(extract_dir)
except Exception as e:
print(f"[ERROR] Failed to extract backup.zip: {e}")
input("Press Enter to exit...")
sys.exit(1)
db_path = extract_dir / "nova.db"
return db_path
# ------------------------------------------------------------------------
# Local source selection
# ------------------------------------------------------------------------
def use_local_source():
print("--- Local Database Selection ---")
print(f'Enter the path to nova.db or a .novabackup or .zip file.')
print(f'Default: {WORK_DIR / "backup_extracted" / "nova.db"}')
print()
while True:
user_db = prompt("Path to database or backup file [Enter for default]")
if not user_db:
db_path = WORK_DIR / "backup_extracted" / "nova.db"
else:
user_db = user_db.strip().strip('"')
db_path = Path(user_db).resolve()
if not db_path.exists():
print(f"[ERROR] File not found: '{db_path}'")
retry = prompt("Try again? [Y/n]", "y")
if retry.lower().startswith("n"):
sys.exit(1)
continue
ext = db_path.suffix.lower()
if ext == ".novabackup":
return extract_local_archive(db_path, restore_prompt=True)
elif ext == ".zip":
return extract_local_archive(db_path, restore_prompt=False)
else:
return db_path
def extract_local_archive(archive_path, restore_prompt=False):
extract_dir = WORK_DIR / "backup_extracted"
if extract_dir.exists():
shutil.rmtree(extract_dir)
print(f'Extracting "{archive_path}" to "{extract_dir}" ...')
try:
with zipfile.ZipFile(archive_path, "r") as zf:
zf.extractall(extract_dir)
except Exception as e:
print(f"[ERROR] Failed to extract archive: {e}")
input("Press Enter to exit...")
sys.exit(1)
db_path = extract_dir / "nova.db"
if restore_prompt and yes_no("Restore this backup in Nova on the connected device?"):
restore_backup_to_device(db_path)
return db_path
def restore_backup_to_device(db_path):
if not ADB_EXE.exists():
print(f"[ERROR] adb.exe not found at '{ADB_EXE}'; cannot send backup to device.")
input("Press Enter to exit...")
sys.exit(1)
devices = adb_devices()
if not devices:
print("[ERROR] No connected ADB device found; cannot restore the backup.")
input("Press Enter to exit...")
sys.exit(1)
serial = devices[0]
nova_backup_dir = "/storage/emulated/0/Android/data/com.teslacoilsw.launcher/files/Backup"
print(f'Copying "{db_path}" to Nova backup folder on {serial} ...')
adb_shell(serial, f"mkdir -p '{nova_backup_dir}'")
res = adb_push(serial, db_path, nova_backup_dir + "/")
if res.returncode != 0:
print("[ERROR] Failed to copy the backup to the device.")
input("Press Enter to exit...")
sys.exit(1)
print(f"Launching Nova Launcher settings on {serial} ...")
run([
str(ADB_EXE), "-s", serial, "shell",
"am", "start",
"-n",
"com.teslacoilsw.launcher/com.teslacoilsw.launcher.preferences.SettingsActivity"
])
print("In Nova, choose Backup & restore and restore the backup you copied.")
# ------------------------------------------------------------------------
# SQLite helpers (use Python sqlite3 module, fall back to CLI)
# ------------------------------------------------------------------------
def sqlite_query(db_path, sql, params=()):
"""Execute a SELECT query using Python sqlite3. Returns list of tuples."""
try:
conn = sqlite3.connect(str(db_path))
cur = conn.execute(sql, params)
rows = cur.fetchall()
conn.close()
return rows
except sqlite3.Error as e:
print(f"[ERROR] SQLite query failed: {e}")
input("Press Enter to exit...")
sys.exit(1)
def sqlite_query_cli(db_path, sql, params=()):
"""Fallback: write SQL to temp file and run sqlite3 CLI."""
sql_file = TEMP / "adbfolder_query.sql"
sql_file.write_text(sql, encoding="utf-8")
res = run([str(SQLITE_EXE), str(db_path)], input=sql, text=True)
if res.returncode != 0:
print(f"[ERROR] sqlite3 CLI failed: {res.stderr}")
input("Press Enter to exit...")
sys.exit(1)
rows = []
for line in res.stdout.strip().splitlines():
rows.append(tuple(line.split("\t")))
return rows
# ------------------------------------------------------------------------
# Target enumeration
# ------------------------------------------------------------------------
class Target:
def __init__(self, num, label, where, order):
self.num = num
self.label = label
self.where = where
self.order = order
def enumerate_targets(db_path):
targets = []
tgt_count = 0
listed_folder_ids = set()
# 1. Dock
print("dock")
tgt_count += 1
targets.append(Target(tgt_count, "dock apps", "f.container = -101",
"f.screen, f.cellY, f.cellX, f.rank, f._id"))
print(f" [{tgt_count}] dock apps")
rows = sqlite_query(db_path,
"SELECT _id, COALESCE(NULLIF(TRIM(title), ''), '(Untitled Folder)') "
"FROM favorites WHERE itemType = 2 AND container = -101 "
"ORDER BY cellY, cellX, rank, _id")
for fid, fname in rows:
tgt_count += 1
label = f"dock - {fname}"
targets.append(Target(tgt_count, label, f"f.container = {fid}",
"f.rank, f._id"))
listed_folder_ids.add(str(fid))
print(f" [{tgt_count}] {fname}")
# 2. Panes 0-14
for pane in range(15):
tgt_count = scan_pane(db_path, pane, targets, tgt_count, listed_folder_ids)
# 3. Other folders (orphaned folders not under dock or a pane)
rows = sqlite_query(db_path,
"SELECT _id, COALESCE(NULLIF(TRIM(title), ''), '(Untitled Folder)') "
"FROM favorites WHERE itemType = 2 "
"ORDER BY LOWER(title), _id")
other_has_folders = False
for fid, fname in rows:
if str(fid) in listed_folder_ids:
continue
if not other_has_folders:
print("other")
other_has_folders = True
tgt_count += 1
label = f"other - {fname}"
targets.append(Target(tgt_count, label, f"f.container = {fid}",
"f.rank, f._id"))
print(f" [{tgt_count}] {fname}")
return targets, tgt_count
def scan_pane(db_path, pane, targets, tgt_count, listed_folder_ids):
rows = sqlite_query(db_path,
"SELECT _id, COALESCE(NULLIF(TRIM(title), ''), '(Untitled Folder)') "
"FROM favorites WHERE itemType = 2 AND container = -100 "
"AND CAST(screen AS INTEGER) = ? "
"ORDER BY cellY, cellX, rank, _id", (pane,))
loose_cnt = sqlite_query(db_path,
"SELECT COUNT(*) FROM favorites WHERE container = -100 "
"AND CAST(screen AS INTEGER) = ? AND itemType <> 2", (pane,))[0][0]
has_pane_folders = len(rows) > 0
print_header = has_pane_folders or loose_cnt > 0
if print_header:
print(f"pane {pane}")
for fid, fname in rows:
tgt_count += 1
label = f"pane {pane} - {fname}"
targets.append(Target(tgt_count, label, f"f.container = {fid}",
"f.rank, f._id"))
listed_folder_ids.add(str(fid))
print(f" [{tgt_count}] {fname}")
if loose_cnt > 0:
tgt_count += 1
label = f"pane {pane} loose apps"
where = (f"f.container = -100 AND CAST(f.screen AS INTEGER) = {pane} "
f"AND f.itemType <> 2")
order = "f.cellY, f.cellX, f.rank, f._id"
targets.append(Target(tgt_count, label, where, order))
print(f" [{tgt_count}] pane {pane} loose apps")
return tgt_count
# ------------------------------------------------------------------------
# Sanitize label -> safe filename
# ------------------------------------------------------------------------
def sanitize_label(label):
safe = re.sub(r'[\\/:*?"<>|() ]+', '_', label).strip('_')
if not safe:
safe = "nova_target"
return safe
# ------------------------------------------------------------------------
# Run query and format output
# ------------------------------------------------------------------------
def run_query(db_path, target, work_dir):
safe_name = sanitize_label(target.label)
raw_out = work_dir / "output.txt"
clean_out = work_dir / f"{safe_name}_apps.txt"
log_out = work_dir / "adbfolder_log.txt"
sql = f"""
SELECT f._id AS item_id,
f.title AS display_name,
CASE
WHEN f.iconPackage IS NOT NULL AND LENGTH(TRIM(f.iconPackage)) > 0
THEN TRIM(f.iconPackage)
WHEN INSTR(f.intent, 'component=') > 0 THEN
CASE
WHEN INSTR(SUBSTR(f.intent, INSTR(f.intent, 'component=') + 10), '/') > 0
THEN SUBSTR(SUBSTR(f.intent, INSTR(f.intent, 'component=') + 10),
1, INSTR(SUBSTR(f.intent, INSTR(f.intent, 'component=') + 10), '/') - 1)
WHEN INSTR(SUBSTR(f.intent, INSTR(f.intent, 'component=') + 10), ';') > 0
THEN SUBSTR(SUBSTR(f.intent, INSTR(f.intent, 'component=') + 10),
1, INSTR(SUBSTR(f.intent, INSTR(f.intent, 'component=') + 10), ';') - 1)
ELSE SUBSTR(f.intent, INSTR(f.intent, 'component=') + 10)
END
WHEN INSTR(f.intent, 'package=') > 0 THEN
CASE
WHEN INSTR(SUBSTR(f.intent, INSTR(f.intent, 'package=') + 8), ';') > 0
THEN SUBSTR(SUBSTR(f.intent, INSTR(f.intent, 'package=') + 8),
1, INSTR(SUBSTR(f.intent, INSTR(f.intent, 'package=') + 8), ';') - 1)
ELSE SUBSTR(f.intent, INSTR(f.intent, 'package=') + 8)
END
ELSE ''
END AS package_name,
f.intent
FROM favorites f
WHERE {target.where}
ORDER BY {target.order};
"""
print()
print(f'Querying "{target.label}" ...')
try:
rows = sqlite_query(db_path, sql)
except SystemExit:
rows = sqlite_query_cli(db_path, sql)
# Write raw tab-delimited output (all columns)
with open(raw_out, "w", encoding="utf-8") as f:
f.write("item_id\tdisplay_name\tpackage_name\tintent\n")
for row in rows:
f.write("\t".join(str(c) if c is not None else "" for c in row) + "\n")
# Format only package_name and display_name, in that order
display_rows = [(row[2], row[1]) for row in rows] # (package_name, display_name)
table_text = ""
if display_rows:
table_text = tabulate(
display_rows,
headers=["package_name", "display_name"],
tablefmt="plain"
)
indented = [" " + line for line in table_text.splitlines()]
# Header matching batch format
if target.label.startswith("pane ") and " - " in target.label:
parts = target.label.split(" - ", 1)
pane_num = parts[0].replace("pane ", "").strip()
folder_name = parts[1]
header = f' pane {pane_num} folder "{folder_name}"'
else:
header = f" {target.label}"
clean_text = header + "\n" + "\n".join(indented) + "\n"
# Write clean output
clean_out.write_text(clean_text, encoding="utf-8")
# Append to running log
timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
entry = (
"\n============================================================\n"
f"Export: {timestamp}\n"
f"{clean_text}"
)
with open(log_out, "a", encoding="utf-8") as f:
f.write(entry)
print()
print(f"--- Result ({clean_out}) ---")
print()
print(clean_text)
print(f"Raw tab-delimited output saved to: {raw_out}")
print(f"Formatted app list saved to: {clean_out}")
print(f"Running export log saved to: {log_out}")
print()
return clean_out
# ------------------------------------------------------------------------
# Optional: check missing third-party packages
# ------------------------------------------------------------------------
def check_missing_3p(serial, db_path, work_dir):
print()
if not yes_no("Check user apps -3 with no icon in Nova before quitting?"):
return
if not ADB_EXE.exists():
print(f"[INFO] adb.exe not found at '{ADB_EXE}'; skipping device package check.")
return
devices = adb_devices()
if not devices:
print("[INFO] No connected ADB device found; skipping device package check.")
return
target_serial = devices[0]
debug(f"RUN_ADB_TARGET={target_serial}")
if not db_path or not db_path.exists():
print("[INFO] Nova database not available; skipping device package check.")
return
resolve_labels = yes_no(
"Resolve app titles via adb/aapt slower over Wi-Fi?",
default="n"
)
do_resolve = resolve_labels
pm_3p_file = TEMP / "adbfolder_pm3.txt"
nova_pkgs_file = TEMP / "adbfolder_nova_pkgs.txt"
unplaced_out = work_dir / "unplaced_user_apps.txt"
print(f"Querying third-party packages from device {target_serial} ...")
res = adb_shell(target_serial, "pm list packages -3 2>/dev/null")
pm_3p_file.write_text(res.stdout, encoding="utf-8")
installed3 = set()
for line in pm_3p_file.read_text(encoding="utf-8").splitlines():
p = line.replace("\r", "").strip()
if p.startswith("package:"):
p = p[8:].strip()
if p:
installed3.add(p)
print("Querying packages referenced anywhere in nova.db ...")
rows = sqlite_query(db_path,
"SELECT DISTINCT package_name FROM ("
" SELECT CASE"
" WHEN iconPackage IS NOT NULL AND LENGTH(TRIM(iconPackage)) > 0"
" THEN TRIM(iconPackage)"
" WHEN INSTR(intent, 'component=') > 0 THEN"
" CASE"
" WHEN INSTR(SUBSTR(intent, INSTR(intent, 'component=') + 10), '/') > 0"
" THEN SUBSTR(SUBSTR(intent, INSTR(intent, 'component=') + 10),"
" 1, INSTR(SUBSTR(intent, INSTR(intent, 'component=') + 10), '/') - 1)"
" ELSE SUBSTR(intent, INSTR(intent, 'component=') + 10)"
" END"
" ELSE ''"
" END AS package_name"
" FROM favorites"
")"
"WHERE LENGTH(package_name) > 0")
in_nova = set()
for (pkg,) in rows:
if pkg:
in_nova.add(pkg)
missing = sorted(installed3 - in_nova, key=str.lower)
results = []
for i, pkg in enumerate(missing, 1):
label = ""
if do_resolve:
if i % 10 == 1 or i == len(missing):
print(f"Resolving titles: {i}/{len(missing)} ({pkg})...")
dump = adb_shell(
target_serial,
f"dumpsys package {pkg} 2>/dev/null"
)
for dl in dump.stdout.splitlines():
m = re.search(r'nonLocalizedLabel="([^"]+)"', dl)
if m:
label = m.group(1).strip()
if not label:
# Try aapt path
# apk_path = shutil.which("aapt") or shutil.which("aapt2")
apk_path = which("aapt") or which("aapt2")
if apk_path:
path_res = adb_shell(
target_serial,
f"pm path {pkg} 2>/dev/null"
)
for pl in path_res.stdout.splitlines():
pl = pl.strip()
if pl.startswith("package:"):
apk_remote = pl[8:].strip()
apk_local = TEMP / "adbfolder_probe.apk"
pull_res = adb_pull(
target_serial,
apk_remote,
apk_local
)
if pull_res.returncode == 0 and apk_local.exists():
badging = run([
apk_path,
"dump",
"badging",
str(apk_local)
])
if badging.stdout:
for bl in badging.stdout.splitlines():
m = re.match(
r"^application-label:'([^']+)'",
bl
)
if m:
label = m.group(1).strip()
break
if badging.returncode != 0:
debug(
f"AAPT failed for {pkg}: {badging.stderr}"
)
try:
apk_local.unlink()
except Exception:
pass
break
results.append((pkg, label))
# Write output
lines = []
if results:
lines.append(
tabulate(
results,
headers=["package_name", "app_title"],
tablefmt="plain"
).splitlines()
)
flat = []
for group in lines:
flat.extend(group)
table_text = "\n".join(flat) + ("\n" if flat else "")
unplaced_out.write_text(table_text, encoding="utf-8")
print()
print(
f"User packages (-3): {len(installed3)} | "
f"In Nova layout: {len(in_nova)} | "
f"No icon in Nova: {len(missing)}"
)
if results:
print()
print(table_text.rstrip())
print(f"Saved unplaced user app list to: {unplaced_out}")
print()
# Cleanup temp files unless debug
if not DEBUG:
for f in [pm_3p_file, nova_pkgs_file]:
try:
f.unlink()
except Exception:
pass
# ------------------------------------------------------------------------
# Main
# ------------------------------------------------------------------------
def main():
global DEBUG
# Allow env override
if os.environ.get("ADBFOLDER_DEBUG", "0") == "1":
DEBUG = True
check_dependencies()
print("Select source for Nova Launcher database:")
print("[1] Pull .novabackup from Android via ADB and extract nova.db [Default]")
print("[2] Use an existing nova.db or .novabackup or .zip already on Windows")
print()
src_choice = prompt("Enter choice [1 or 2]", "1")
if src_choice == "2":
db_path = use_local_source()
else:
db_path = pull_from_android()
if not db_path or not db_path.exists():
print(f"[ERROR] nova.db not found at: '{db_path}'")
input("Press Enter to exit...")
sys.exit(1)
print()
print(f"Using database: {db_path}")
print("Scanning Nova Launcher dock, panes and folders ...")
print()
targets, tgt_count = enumerate_targets(db_path)
print()
pick = prompt(f"Select number [1-{tgt_count}, q to quit]")
if pick.lower() == "q" or pick == "0":
check_missing_3p(None, db_path, WORK_DIR)
print("Bye.")
return
try:
idx = int(pick) - 1
if not (0 <= idx < tgt_count):
raise ValueError
except ValueError:
print(f"[ERROR] Invalid selection: '{pick}'")
input("Press Enter to exit...")
sys.exit(1)
target = targets[idx]
debug(f"SELECTED_LABEL={target.label}")
debug(f"SQL_WHERE={target.where}")
debug(f"SQL_ORDER={target.order}")
run_query(db_path, target, WORK_DIR)
while True:
loop_more = yes_no("Export another folder or pane?", default="n")
if not loop_more:
break
# Re-enumerate (in case something changed) and pick again
targets, tgt_count = enumerate_targets(db_path)
print()
pick = prompt(f"Select number [1-{tgt_count}, q to quit]")
if pick.lower() == "q" or pick == "0":
break
try:
idx = int(pick) - 1
if not (0 <= idx < tgt_count):
raise ValueError
except ValueError:
print(f"[ERROR] Invalid selection: '{pick}'")
break
target = targets[idx]
run_query(db_path, target, WORK_DIR)
check_missing_3p(None, db_path, WORK_DIR)
input("Press Enter to exit...")
if __name__ == "__main__":
main()
# ----------------------------------------------------------------------
# end of adbfolder.py
# ----------------------------------------------------------------------
# ----------------------------------------------------------------------
# Step by step description of what this script does
# ----------------------------------------------------------------------
# 1. Environment & Configuration Setup
# A. Command environment
# a. Initializes Python execution environment
# b. Imports required Python modules and dependencies
# B. Configuration variables
# a. Sets default configuration paths for ADB, SQLite
# and work directories
# b. Defines the DEBUG toggle flag
# c. Defines temporary and Android backup directories
#
# 2. Dependency Verification
# A. Executable checks
# a. Locates adb.exe and checks system PATH
# or configured fallback path
# b. Locates sqlite3.exe and exits with an error if missing
# B. Directory setup
# a. Creates the Nova work directory if it does not exist
#
# 3. Source Selection Menu
# A. Choice [1]: Pull from Android via ADB
# a. Detects connected ADB device target
# b. Optionally launches Nova Launcher settings to create a backup
# c. Lists .novabackup files in the remote backup directory
# with fallback search of Android storage
# d. Prompts user to select or enter a backup filename/path
# e. Pulls the selected .novabackup file to the Windows workspace
# f. Renames the pulled file to backup.zip and extracts it
# into backup_extracted to retrieve nova.db
# B. Choice [2]: Use Local Database / Archive
# a. Prompts user for a custom path to nova.db, .novabackup or .zip
# with a retry loop
# b. Validates file existence
# c. Automatically extracts archives when necessary
# d. Optionally offers to restore a .novabackup database
# to the device
#
# 4. Database Verification & Target Enumeration
# A. Database validation
# a. Verifies that nova.db exists at the target path
# B. Section 1 [Dock]
# a. Scans dock apps [container = -101]
# b. Scans and lists any dock folders
# C. Section 2 [Panes 0-14]
# a. Iterates through home screen panes 0 to 14
# via the scan_pane function
# b. Queries pane folders [itemType = 2, container = -100]
# c. Checks for loose non-folder apps [itemType <> 2] on each pane
# D. Section 3 [Other Folders]
# a. Searches for orphaned folders not categorized under dock
# or panes
# b. Lists and registers remaining folders
#
# 5. User Selection & Query Execution [run_query]
# A. Selection & Sanity
# a. Prompts user to select a target number or quit [q or 0]
# b. Sanitizes the selected label to generate a safe filename
# B. Query Generation & Execution
# a. Builds a dynamic SQLite query extracting:
# i. Item ID
# ii. Display name
# iii. Package name
# iv. Raw intent
# b. Executes the query against nova.db using Python sqlite3
# c. Falls back to the sqlite3 command-line executable if necessary
# C. Output & Formatting
# a. Writes raw tab-delimited results to output.txt
# b. Formats output columns [package_name, display_name]
# into an aligned table via tabulate
# c. Adds a descriptive pane/folder heading and indents the table
# by two spaces for easier reading and future concatenation
# d. Saves formatted results to [target]_apps.txt
# e. Appends each formatted export with a timestamp to
# adbfolder_log.txt without overwriting previous exports
# f. Displays the formatted results and output file locations
# D. Repeat Export
# a. Prompts user to export another folder or pane
# b. Re-enumerates targets in case the database layout has changed
# c. Repeats the selection and export process as requested
#
# 6. Optional Unplaced Third-Party Package Check [check_missing_3p]
# A. Pre-check Validation
# a. Prompts user whether to check for user-installed packages
# [-3] missing from the Nova layout
# b. Verifies ADB device connectivity and database availability
# c. Optionally prompts to resolve app titles via adb/aapt
# B. Data Collection
# a. Queries installed third-party packages from device
# [adb shell pm list packages -3]
# b. Queries all referenced packages inside nova.db
# c. Compares installed packages against packages referenced in Nova
# C. Optional App Title Resolution
# a. Runs adb shell dumpsys package for each unplaced package
# b. Searches for nonLocalizedLabel in the package information
# c. If unavailable, uses adb shell pm path to locate the APK
# d. Pulls the APK to the PC and uses aapt/aapt2 to read its label
# D. Diff Analysis & Output
# a. Identifies installed third-party packages with no Nova icon
# b. Generates a formatted package_name / app_title table
# c. Saves results to unplaced_user_apps.txt
# d. Displays package counts and the resulting unplaced app list
# E. Cleanup
# a. Removes temporary package-list files unless DEBUG=1
# b. Exits cleanly
# -------------------------------------------------------------------
# -------------------------------------------------------------------
--
Usenet is all about intelligent kind-hearted people helping the team learn.
--- Synchronet 3.22a-Linux NewsLink 1.2