Home-Databases/Aliexpress-Part-DB-orders-pull
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Upload files to "/"

Browse Source
This commit is contained in:
Amir
2026-08-16 03:57:24 +02:00
parent 7235a0913c
commit bfe86c416f
5 changed files with 1995 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+503 -2
View File
@@ -1,3 +1,504 @@
# Aliexpress-Part-DB-orders-pull
# AliExpress → Part-DB Importer
Aliexpress-Part-DB-orders-pull
A two-script toolkit for taking an AliExpress `orders.csv` export and turning
it into a Part-DB v2.15.x inventory with **product images automatically
attached to each part**.
```
┌─────────────────┐ ┌──────────────────────────┐ ┌─────────────┐
│ orders.csv │ ─► │ aliexpress_to_partdb.py │ ─► │ partdb_ │
│ (from AliExpr.) │ │ • renames columns │ │ import.csv │
└─────────────────┘ │ • downloads all images │ └──────┬──────┘
│ • writes attachments │ │
│ index │ [import via
└──────────────────────────┘ Part-DB web UI]
│
▼
┌──────────────────────────┐ ┌─────────────┐
│ attach_images_to_ │ ◄──│ parts now │
│ partdb.py │ │ exist in │
│ • reads index │ │ Part-DB │
│ • finds each part by │ └─────────────┘
│ name via REST API │
│ • uploads image as │
│ attachment │
└──────────────────────────┘
```
---
## What's in this package
| File | Purpose |
|---|---|
| `aliexpress_to_partdb.py` | Reads AliExpress `orders.csv`, downloads item images, writes Part-DB-format CSV |
| `attach_images_to_partdb.py` | Pushes the downloaded images to Part-DB and attaches them via the REST API |
| `README.md` | This file |
Put all three files in the same folder, e.g. `C:\Users\Amir\Downloads\aliexpress-import\`.
---
## Prerequisites
1. **Python 3.8 or newer** — check with `python --version`
2. **The `requests` library** — install with:
```cmd
pip install requests
```
3. **Part-DB v2.15.x** running and accessible (test by opening it in your browser)
4. **A Part-DB API token** (see below)
5. **At least one Attachment type** defined in Part-DB (e.g. "Image")
— see [Step 1.5](#step-15-one-time-create-an-image-attachment-type)
---
## One-time setup
### Step 1: Get an API token in Part-DB
1. Open Part-DB in your browser, log in
2. Click your **username** (top right) → **Settings**
3. In the left sidebar, click **API tokens**
4. Click **+ Create new token** (or "New")
5. Give it any name (e.g. `importer`) and click **Save**
6. **Copy the token** — it looks like `tcp_xxxxxxxxxxxxxxxxxxxxxxxxxx`
- You won't be able to see it again after this dialog closes
- If you lose it, just create a new one
> If the "API tokens" section is missing, your user doesn't have the
> **API** permission. If you installed Part-DB yourself you're an admin
> and already have it; otherwise ask your admin to enable
> "Miscellaneous → API access" + "Create API tokens" for your user.
### Step 1.5 (one-time): Create an "Image" attachment type
Part-DB requires an attachment type to exist before any attachment can be created.
1. In Part-DB, click **Admin** in the left sidebar
2. Click **Attachment types**
3. Click **+ New**
4. Name: `Image` (or whatever you prefer)
5. Filetypes: leave blank, or add `image/jpeg`, `image/png` for stricter validation
6. Click **Save**
Verify it worked:
```cmd
python attach_images_to_partdb.py --list-attachment-types
```
You should see your "Image" type listed.
### Step 2: Configure the script
Open `attach_images_to_partdb.py` in any text editor and update the two
lines near the top:
```python
DEFAULT_PARTDB_URL = "http://YOUR-PART-DB:PORT" # e.g. http://192.168.64.80:8085
DEFAULT_PARTDB_TOKEN = "tcp_YOUR_TOKEN_HERE" # the token you just copied
```
> **Security**: this file now contains a secret. Treat it like a password.
> See the [Security](#security) section at the bottom for the full list
> of dos and don'ts.
---
## The full workflow (for your first import)
### Step A: Export your orders from AliExpress
1. Go to [aliexpress.com](https://www.aliexpress.com) → **My Orders**
2. Click **Export** (top-right of the orders table) → choose **CSV**
3. Save the file as `orders.csv` in the same folder as the scripts
(e.g. `C:\Users\Amir\Downloads\aliexpress-import\orders.csv`)
### Step B: Run the conversion script
```cmd
cd C:\Users\Amir\Downloads\aliexpress-import
python aliexpress_to_partdb.py orders.csv
```
This produces, in the same folder:
- `partdb_import.csv` — the Part-DB-format CSV (one row per item)
- `partdb_images/` — folder of all downloaded product images
- `partdb_images/attachments.json` — index of which image goes with which part
You should see output like:
```
Loaded 553 order line(s).
All done.
CSV written to: C:\...\partdb_import.csv
Images stored in: C:\...\partdb_images
Images downloaded: 553/553 (0 failed)
```
### Step C: Import the CSV into Part-DB
1. Open Part-DB in your browser
2. In the left sidebar, click **Tools** → **Import parts** (or similar)
3. Click **Choose file** → select `partdb_import.csv`
4. **Important settings** in the import wizard:
- ✅ **CSV delimiter**: `,` (comma — the script uses standard CSV)
- ✅ **Create unknown datastructures**: **checked** (so the "New parts" category and any new suppliers are auto-created)
- **Mode**: "Create new entries" (default) — see [FAQ](#faq) for re-import behavior
5. Click through the column mapping. The headers in `partdb_import.csv` are
already named to match Part-DB's import schema, so the auto-map should
fill in everything correctly
6. Click **Import** / **Start import**
7. Wait for the progress bar to finish
> ⚠️ **Watch out for re-imports** (see [FAQ](#faq) below). The default
> "Create" mode will make duplicates if you re-import the same CSV.
### Step D: Attach the images
```cmd
cd C:\Users\Amir\Downloads\aliexpress-import
python attach_images_to_partdb.py
```
You'll see one line per upload, e.g.:
```
[ 1/553] ENS160 ENS160+AHT21 CARBON Dioxide CO2 e OK part 558: attached to part 558
[ 2/553] TPS63020 Micro Power Module Automatic Vo OK part 559: attached to part 559
...
----------------------------------------------------------------------
Done.
Uploaded: 553
Skipped: 0 (already attached)
No match: 0 (part not found in Part-DB)
Failed: 0
```
For ~553 items at 0.15s between calls, expect roughly **1–2 minutes**.
The script is **idempotent** — running it again will skip parts that
already have an attachment with the same filename. So you can re-run it
anytime without creating duplicates.
### Step E: Verify
Open any of the imported parts in Part-DB — the product image should
appear in the **Attachments** tab (and as a thumbnail in the part list
if Part-DB auto-set it as the preview, which it usually does).
---
## For future use (when you buy more parts)
The next time you want to import a fresh AliExpress order:
1. Export the new `orders.csv` from AliExpress
2. If the new CSV contains only new parts (no overlap with the old import),
you can just re-run **Steps B → D** above with the new file
3. If the new CSV might overlap with the old one, read the
[Re-imports / dedup](#faq) section in the FAQ below
**Optional but recommended**: rotate your Part-DB API token periodically
to keep it from accumulating exposure. Just delete the old one in
Part-DB's API tokens page and paste the new one into the script.
---
## Script 1: `aliexpress_to_partdb.py`
### What it does
1. Reads `orders.csv` from AliExpress
2. Maps each row to a Part-DB part using the v2.15.x CSV import schema
3. Downloads every `Item image url` to `partdb_images/` (upgrades
AliExpress's `_220x220.jpg` thumbnail URLs to full-size, falls back
to the thumbnail on 404)
4. Writes `attachments.json` mapping each part to its image file(s)
5. Writes `partdb_import.csv` ready for the Part-DB import wizard
### Run command
```cmd
cd C:\Users\Amir\Downloads\aliexpress-import
python aliexpress_to_partdb.py orders.csv
```
### Options
```cmd
python aliexpress_to_partdb.py orders.csv -o my_parts.csv -d ./images
python aliexpress_to_partdb.py orders.csv --no-images :: CSV only, skip downloads
```
### Customizing the default category
The default category for all imported parts is `New parts`. To change it
(for example to `Electronics/Sensors` or `Imported/2026-Q3`):
1. Open `aliexpress_to_partdb.py` in a text editor
2. Find the line near the top:
```python
DEFAULT_CATEGORY = "New parts"
```
3. Change the string to whatever you want
4. Save the file
> The category is **auto-created** the first time you import, as long as
> "Create unknown datastructures" is checked in the Part-DB import wizard
> (see Step C above). Sub-categories use `->` as the separator
> (e.g. `Imported -> 2026 -> Q3`).
### Output files
| File | Contents |
|---|---|
| `partdb_import.csv` | The CSV you upload to Part-DB's import wizard |
| `partdb_images/<part-name>_<idx>.jpg` | One image per part, full size when available |
| `partdb_images/attachments.json` | Index: row → part name → image file path. Used by script 2 |
---
## Script 2: `attach_images_to_partdb.py`
### What it does
1. Reads `attachments.json` from script 1
2. For each entry, queries Part-DB for a part with the matching `name`
3. Auto-picks an attachment type whose name looks like "Image" /
"Photo" / etc., or falls back to the first available type
4. Uploads the image as a PartAttachment
5. Skips parts that already have an attachment with the same filename
(idempotent — safe to re-run)
### Run command (just run it)
```cmd
cd C:\Users\Amir\Downloads\aliexpress-import
python attach_images_to_partdb.py
```
The URL and token are read from the `DEFAULT_PARTDB_URL` and
`DEFAULT_PARTDB_TOKEN` constants at the top of the file. Override with
env vars (recommended for shared machines) or CLI flags (see below).
### Options
```cmd
:: Show all flags
python attach_images_to_partdb.py --help
:: Override the URL/token (e.g. for a different Part-DB instance)
python attach_images_to_partdb.py --url http://localhost:8000 --token tcp_xxx
:: Test with the first 3 entries, no changes made
python attach_images_to_partdb.py --limit 3 --dry-run
:: Run for real, only on the first 50
python attach_images_to_partdb.py --limit 50
:: List all attachment types in your Part-DB (useful for picking one)
python attach_images_to_partdb.py --list-attachment-types
:: Force a specific attachment type
python attach_images_to_partdb.py --attachment-type /api/attachment_types/1
:: Skip TLS verification (self-signed certs)
python attach_images_to_partdb.py --insecure
```
### Using env vars instead of editing the file (more secure)
```cmd
set PARTDB_URL=http://192.168.64.80:8085
set PARTDB_TOKEN=tcp_xxxxxxxx
python attach_images_to_partdb.py
```
In PowerShell:
```powershell
$env:PARTDB_URL = "http://192.168.64.80:8085"
$env:PARTDB_TOKEN = "tcp_xxxxxxxx"
python attach_images_to_partdb.py
```
---
## FAQ
### When I re-import the same CSV, does it skip existing parts?
**No, by default Part-DB will create duplicates.** The CSV importer's
default mode is "Create new entries" — it does not check for existing
parts.
For the AliExpress use case (one big import, plus small future imports
of new items), this is rarely a problem because the new orders contain
different parts. But if you ever need to re-import the same file (or a
file that overlaps with an old one), do one of these:
**Option 1 — Use the import wizard's matching mode**
In the Part-DB import wizard, look for a "Mode" or "Match existing
parts" option:
- **Create** (default) — always creates new parts → duplicates
- **Update** — updates matching parts, leaves the rest alone
- **Skip** — skips any part that already matches
If available, pick **Update** or **Skip** and choose the match key
(usually `name` works, but `manufacturer_product_number` is more
reliable for electronics).
**Option 2 — Delete duplicates manually after import**
After the import, in the Part-DB parts list, sort by name, find the
duplicates, select all → **Delete**. Tedious but works.
**Option 3 — Import to a temporary category, then move**
Import into a unique category (e.g. `Incoming/2026-08-15`), then
manually move parts to their final categories. Anything that was
already in the final categories will now appear twice, which is easy
to spot.
### The image script says "no attachment types defined"
You skipped [Step 1.5](#step-15-one-time-create-an-image-attachment-type)
or your Part-DB has no attachment types at all. Fix:
```cmd
python attach_images_to_partdb.py --list-attachment-types
```
If it prints nothing, go to **Admin → Attachment types** in Part-DB's
web UI and create at least one (recommended name: `Image`).
### The image script says "401 Unauthorized"
Token is wrong, expired, or your user lacks API permission.
- Open Part-DB → User menu → Settings → API tokens → create a new one
- Paste it into `DEFAULT_PARTDB_TOKEN` in the script (or set
`PARTDB_TOKEN` env var)
- If the API tokens section is missing, ask your admin to grant
"Miscellaneous → API access" + "Create API tokens" to your user
### The image script says "no match in Part-DB" for some rows
Either:
- You skipped Step C (CSV import) — the parts aren't in Part-DB yet
- The part name in Part-DB differs slightly from what's in the CSV
(e.g. truncated, case-different, or with extra characters)
- You're looking at parts in a different Part-DB instance
The first one is most common. Run Step C, then re-run the image script —
it will skip already-uploaded ones and pick up where it left off.
### The connection fails / "cannot reach Part-DB"
- Make sure `http://YOUR-PART-DB:PORT` opens in your browser first
- If Part-DB is on HTTPS, change `DEFAULT_PARTDB_URL` to start with
`https://` and possibly add `--insecure` if you have a self-signed cert
- If Part-DB is on a different machine, check the firewall / port
forwarding
### Can I import other vendors (LCSC, Mouser, Digi-Key)?
The CSV format from other vendors is different. To support them you'd
need a new conversion script. The `aliexpress_to_partdb.py` script has
the AliExpress header names hard-coded near the top — you'd write a
parallel script with a different `AE = {...}` mapping. Happy to help if
you need this.
### How do I move all my AliExpress parts to a different category later?
In Part-DB, go to the part list, filter by `category = "New parts"`,
select all → **Edit** → change the category. Or use the Part-DB REST
API for bulk moves.
---
## Troubleshooting checklist
| Symptom | Likely cause | Fix |
|---|---|---|
| `File invalid. Please check that you have selected the right format!` in Part-DB import | CSV column names don't match Part-DB's schema | Re-run `aliexpress_to_partdb.py`; the current version uses the verified 2.15.x column names |
| `must be one of Orderdetail[] (string given)` | Old script version used `orderdetails` as a flat column | Use the latest `aliexpress_to_partdb.py` from this package |
| `KeyError: 'ordernumber'` at script startup | Old script version | Use the latest version from this package |
| `DeprecationWarning: 'maxsplit' is passed as positional argument` | Old script version (cosmetic) | Update to the latest version |
| Image script: `no attachment types` | Skipped Step 1.5 | See [this FAQ entry](#the-image-script-says-no-attachment-types-defined) |
| Image script: `no match in Part-DB` | Skipped CSV import step, or names differ | Run Step C first, or check Part-DB web UI |
| Image script: `401 Unauthorized` | Bad/expired token, or no API permission | Recreate token in Part-DB; check user has API permission |
| Image script: `SSL: CERTIFICATE_VERIFY_FAILED` | Self-signed HTTPS cert | Add `--insecure` flag |
| Some images fail to download (`404 Client Error`) | Seller removed the image from AliExpress CDN | Script falls back to the thumbnail; if even that fails, the part is created without an image — you can attach a manual photo later |
---
## Security
This package handles an API token that grants access to your Part-DB
under your user account. Please:
✅ **Do:**
- Keep the script folder on your local machine only
- Use a fresh token dedicated to this purpose
- Treat the file like a password — it should not leak
- Rotate (delete + recreate) the token periodically
- If you ever share your screen / codebase, blank the
`DEFAULT_PARTDB_TOKEN` line first
- For a "cleaner" approach, use the `PARTDB_TOKEN` env var instead of
baking it into the file
❌ **Don't:**
- Commit the folder to git (add a `.gitignore` containing `*.py` if you must)
- Sync the folder to cloud backups (OneDrive, Dropbox, Google Drive,
iCloud) without first removing the token
- Share the folder publicly
- Re-use the same token for other applications
- Leave the file on a USB stick you might lose
### If your token leaks
1. In Part-DB: User menu → Settings → API tokens → delete the leaked one
2. Create a new token and paste it into the script
3. Check Part-DB's audit log for unexpected activity
4. Done — the new token works immediately
---
## Files at a glance
```
aliexpress-import\
├── README.md ← you are here
├── aliexpress_to_partdb.py ← script 1: CSV + image downloader
├── attach_images_to_partdb.py ← script 2: REST API uploader
├── orders.csv ← (your input) AliExpress export
├── partdb_import.csv ← (output) Part-DB-ready CSV
└── partdb_images\ ← (output) downloaded images
├── ENS160_ENS160+AHT21_..._1.jpg
├── TPS63020_Micro_Power_..._2.jpg
├── …
└── attachments.json ← (output) image index for script 2
```
---
## Quick reference card
| Task | Command |
|---|---|
| Convert CSV + download images | `python aliexpress_to_partdb.py orders.csv` |
| Skip image download (CSV only) | `python aliexpress_to_partdb.py orders.csv --no-images` |
| Import to Part-DB | Use web UI: Tools → Import parts → upload `partdb_import.csv` |
| Attach all images | `python attach_images_to_partdb.py` |
| Test 3 images first | `python attach_images_to_partdb.py --limit 3 --dry-run` |
| List attachment types | `python attach_images_to_partdb.py --list-attachment-types` |
| Show all flags | `python attach_images_to_partdb.py --help` |
---
## Version notes
Tested with:
- AliExpress CSV export (as of mid-2026)
- Part-DB v2.15.0
- Python 3.10 / 3.11 on Windows 10/11
If Part-DB changes its CSV schema in a future version, you may need to
update the column names in `PARTDB_COLUMNS` near the top of
`aliexpress_to_partdb.py`. Check the official docs:
<https://docs.part-db.de/usage/import_export.html>