Mindtunnel.com Projects

Assorted Spare Time Projects

dt: Darktable database management tool

| Comments

dt is a command line utility for managing the Darktable sqlite3 database.

The reason for this utility is that I wanted to move away from using Picasa for my photo management and use Darktable in it’s place. The problem was that I had thousands of photos in Picasa, all captioned, geo-tagged, face-tagged, cropped, rotated, etc, and I wanted a way to get as much of that data into Darktable as possible without having to manually redo everything.

To start, I had to save as much of the data from Picasa as I could. For that I created the picasa3meta library and metaSave utility. With the metaSave utility I was able to extract as much information as possible from Picasa and save it into .meta sidecar files.

Once I had all the metadata from Picasa I moved my all my images to the new directory structure I wanted to use and imported them into Darktable. Darktable does not pick up the metadata however as Picasa does not store it into the original image files and it does not read the .meta sidecar files created by metaSave. Darktable also does not pick up any edits I made in Picasa (the only one I really wanted to transfer was crop&rotate).

Darktable creates a sidecar file (.xmp) that contains the edit history and metadata for each image. I initially thought I could just update that file and Darktable would pick up the changes. But that was not the case - any changes I made there were just discarded by Darktable on startup and replaced by what was stored in the Darktable sqlite3 database (~/.config/darktable/library.db on Ubuntu). After a lot of trial and error I determined I did not need to worry about the sidecar file at all. I could just make whatever changes I wanted to the database (making sure Darktable was not running at the same time of course) and then start Darktable and tell tell it to rewrite the sidecar files.

Here is what dt will do:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
$ dt --help
usage: dt [-h] [-d DTDB] [--no-backup] <command> <parm> [<parm> ...]

Darktable filmroll/image/metadata maintenance:

mv <src> <dest>                             : rename film roll or image
query <dir|file> [ <dir|file> ... ]         : dump details of an object
                                              (best on 132+ column term)
scan <dir|file> [ <dir}|file> ... ]         : just report if the object 
                                              doesn't exist in the db - 
                                              .xmp and .meta extensions 
                                              will be stripped before the 
                                              check
set <image> <var> <val> [ <var> <val> ... ] : modify metadata

For the 'set' command, the valid var's are:

image table:
    datetime_taken         "YYYY:MM:DD HH:MM:SS"
    caption                "text"
    description            "text"
    license                "text"
    longitude              float (-180.0 to +180.0, positive E) (Note 3)
    latitude               float (-90.0 to +90.0, positive N) (Note 3)

meta_data table:
    creator                "text"
    publisher              "text"
    title                  "text"
    description            "text"
    rights                 "text"

history table:
    crop                   "angle,cx,cy,cw,ch" (Note 4)

tags and tagged_images tables:
    tag                    "text" (Note 5)

NOTES:
  1) before modifying the database dt will check to make sure darktable 
     is not running
  2) if a modification is going to be made to the database a backup will 
     be made first: <library.db>.<timestamp>, unless you specify --no-backup
  3) to convert degrees minutes seconds to a float: 
        f = degrees + ( minutes + seconds/60. ) / 60.
     and change sign if W or S.
  4) angle is the rotation angle before crop, positive clockwise,
     cx, cy, cw, ch are the normalized crop coordinates (0.0 to 1.0),
     cx,cy is lower left(?) and cw,ch are new width and height.
  5) you may add tags but not modify or remove existing tags

positional arguments:
  <command>           mv, query, set, etc
  <parm>

optional arguments:
  -h, --help          show this help message and exit
  -d DTDB, --db DTDB  Darktable database path, default=
                      ~/.config/darktable/library.db
  --no-backup         don't backup the Darktable database before modifications

As an example, if you had an image called photo.jpg with the Picasa metadata saved in photo.jpg.meta, you could do something like this to import the metadata into Darktable:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
#!/bin/bash

IM=photo.jpg
CAPTION=$(grep -aF pmp.caption "$IM.meta" | cut -d: -f2-)
LAT=$(grap -aF pmp.lat "$IM.meta" | cut -d: -f2-)
LONG=$(grap -aF pmp.long "$IM.meta" | cut -d: -f2-)
KEYWORD=$(grep -aF ini.sfaces "$IM.meta" | cut -d: -f2-)
TILT=$(grep -aF pmp.filters.tilt "$IM.meta" | cut -d: -f2-)
CROP=$(grep -aF pmp.crop64.xy "$IM.meta" | cut -d: -f2-)

# perform any validity checks on the above vars here ....

dt set "$IM" caption "$CAPTION" latitude $LAT longitude $LONG crop $TILT,$CROP

# I know my Picasa keywords string is a comma separated list, so ...
IFS="," ; for T in $KEYWORD
do
  dt set "$IM" tag "$T"
done
 

Notes

See dt for more details and issues.

Comments