LIS Command Line Tools¶
This describes the command line tools that are available for processing LIS files.
The tools are located in TotalDepth/src/TotalDepth/LIS
Scanning LIS Files¶
These command line tools scan LIS files without changing them.
ScanPhysRec.py¶
Scans a LIS79 file and reports the Physical Record structure.
Arguments¶
One argument that will be treated as a path to a LIS file.
Options¶
| Option | Description |
|---|---|
| --version | Show program’s version number and exit |
| -h, --help | Show this help message and exit. |
| -k, --keep-going | Keep going as far as sensible. [default: False] |
| -l LOGLEVEL, --loglevel=LOGLEVEL | Log Level (debug=10, info=20, warning=30, error=40, critical=50) [default: 20] |
Examples¶
Example of scanning a non-TIF encoded file:
$ python3 ScanPhysRec.py LIS.lis
Cmd: ScanPhysRec.py LIS.lis
PR: tell() Length Attr LD_len RecNum FilNum ChkSum LR Attr [Total LD]
-------------------------------------- start --------------------------------------
PR: 0x 0 62 0x 0 58 ------ ------ ------ 0x80 0x00 [ 58]
PR: 0x 3e 1024 0x 1 1020 ------ ------ ------ 0x22 0x00
PR: 0x 43e 1024 0x 3 1020 ------ ------ ------ + -- --
PR: 0x 83e 1024 0x 3 1020 ------ ------ ------ + -- --
PR: 0x c3e 1024 0x 3 1020 ------ ------ ------ + -- --
PR: 0x 103e 1024 0x 3 1020 ------ ------ ------ + -- --
PR: 0x 143e 1024 0x 3 1020 ------ ------ ------ + -- --
PR: 0x 183e 1024 0x 3 1020 ------ ------ ------ + -- --
PR: 0x 1c3e 1024 0x 3 1020 ------ ------ ------ + -- --
PR: 0x 203e 34 0x 2 30 ------ ------ ------ + -- -- [ 8190]
PR: 0x 2060 304 0x 0 300 ------ ------ ------ 0x40 0x00 [ 300]
PR: 0x 2190 1014 0x 0 1010 ------ ------ ------ 0x00 0x00 [ 1010]
PR: 0x 2586 1014 0x 0 1010 ------ ------ ------ 0x00 0x00 [ 1010]
PR: 0x 297c 1014 0x 0 1010 ------ ------ ------ 0x00 0x00 [ 1010]
PR: 0x 2d72 1014 0x 0 1010 ------ ------ ------ 0x00 0x00 [ 1010]
PR: 0x 3168 1014 0x 0 1010 ------ ------ ------ 0x00 0x00 [ 1010]
PR: 0x 355e 1014 0x 0 1010 ------ ------ ------ 0x00 0x00 [ 1010]
PR: 0x 3954 1014 0x 0 1010 ------ ------ ------ 0x00 0x00 [ 1010]
PR: 0x 3d4a 1014 0x 0 1010 ------ ------ ------ 0x00 0x00 [ 1010]
PR: 0x 4140 942 0x 0 938 ------ ------ ------ 0x00 0x00 [ 938]
PR: 0x 44ee 62 0x 0 58 ------ ------ ------ 0x81 0x00 [ 58]
PR: EOF
--------------------------------------- EOF ---------------------------------------
PR Count: 21
Histogram of Physical Record lengths:
Bytes
34 [1] | +++++++++++
62 [2] | ++++++++++++++++++++++
304 [1] | +++++++++++
942 [1] | +++++++++++
1014 [8] | ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1024 [8] | ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
CPU time = 0.001 (S)
Bye, bye!
First ScanPhysRec.py echo’s the command line then it scans the Physical Records an writes out a table that has the following columns:
| Heading | Description |
|---|---|
| tell() | The file position of the start of the Physical Record as a hex integer. |
| Length | The length of the Physical Record as a decimal integer. |
| Attr | The Physical Record Header attributes as a hex integer. |
| LD_len | The length of the logical data payload contained in this Physical Record. |
| RecNum | A record number from the Physical Record trailer if present, otherwise: ------ |
| FilNum | A file number from the Physical Record trailer if present, otherwise: ------ |
| ChkSum | A checksum from the Physical Record trailer if present, otherwise: ------ |
| LR | Logical Record type from the Logical Record Header as a hex integer. |
| Attr | Logical Record attributes from the Logical Record Header as a hex integer. This is (almost?) always 0x00 |
| [Total LD] | The total length of the logical data in the Logical Record if a terminator Physical Record, otherwise blank. |
This is followed by an ASCII histogram of the lengths of all Physical Records with the following columns:
- The size in bytes.
- The frequency count.
- A series of + that is proportionate to the frequency count.
If TIF markers are detected then the output adds TIF columns thus:
TIF ? : Type Back Next PR: tell() Length Attr LD_len RecNum FilNum ChkSum LR Attr [Total LD]
--------------------------------------------------------------- start ---------------------------------------------------------------
TIF True >: 0x 0 0x 0 0x 4a PR: 0x 0 62 0x 0 58 ------ ------ ------ 0x80 0x00 [ 58]
TIF True >: 0x 0 0x 0 0x 456 PR: 0x 4a 1024 0x 1 1020 ------ ------ ------ 0x22 0x00
TIF True >: 0x 0 0x 4a 0x 862 PR: 0x 456 1024 0x 3 1020 ------ ------ ------ + -- --
TIF True >: 0x 0 0x 456 0x c6e PR: 0x 862 1024 0x 3 1020 ------ ------ ------ + -- --
TIF True >: 0x 0 0x 862 0x 107a PR: 0x c6e 1024 0x 3 1020 ------ ------ ------ + -- --
TIF True >: 0x 0 0x c6e 0x 1486 PR: 0x 107a 1024 0x 3 1020 ------ ------ ------ + -- --
TIF True >: 0x 0 0x 107a 0x 1892 PR: 0x 1486 1024 0x 3 1020 ------ ------ ------ + -- --
TIF True >: 0x 0 0x 1486 0x 1c9e PR: 0x 1892 1024 0x 3 1020 ------ ------ ------ + -- --
TIF True >: 0x 0 0x 1892 0x 20aa PR: 0x 1c9e 1024 0x 3 1020 ------ ------ ------ + -- --
TIF True >: 0x 0 0x 1c9e 0x 20ec PR: 0x 20aa 54 0x 2 50 ------ ------ ------ + -- -- [ 8210]
The additional columns are:
| Heading | Description |
|---|---|
| ? | ? |
| Type | TIF marker type, 0 for in-file record, 1 for EOF. |
| Back | The file position of the precious TIF marker as a hex integer. |
| Next | The file position of the next TIF marker as a hex integer. |
ScanLogiRec.py¶
Scans a LIS79 file and reports the Logical Record structure.
Arguments¶
One argument that will be treated as a path to a LIS file.
Options¶
| Option | Description |
|---|---|
| --version | Show program’s version number and exit |
| -h, --help | Show this help message and exit. |
| -k, --keep-going | Keep going as far as sensible. [default: False] |
| -l LOGLEVEL, --loglevel=LOGLEVEL | Log Level (debug=10, info=20, warning=30, error=40, critical=50) [default: 20] |
| -v, --verbose | Verbose output, this outputs a representation of table data and DFSRs. |
Examples¶
Example of scanning a LIS file:
$ python3 ScanLogiRec.py RW.lis
Cmd: ScanLogiRec.py RW.lis
0x00000000 <TotalDepth.LIS.core.LogiRec.LrFileHeadRead object at 0x1007981d0>: "File header"
2012-02-08 17:43:45,078 WARNING LrTableRead(): Discarding duplicate row b'BS7 ' in table b'CONS'
2012-02-08 17:43:45,087 WARNING LrTableRead.__init__(): Tell: 0x4a LD index: 0x32 Error: FileRead.unpack(): Bytes: b'\x00' not enough for struct that needs: 12 bytes.
0x0000004a <TotalDepth.LIS.core.LogiRec.LrTableRead object at 0x100798210>: "Well site data"
0x000020ec <TotalDepth.LIS.core.LogiRec.LrDFSRRead object at 0x1007981d0>: "Data format specification record"
0x0006141c <TotalDepth.LIS.core.LogiRec.LrFileTailRead object at 0x10058e7d0>: "File trailer"
0x00061466 <TotalDepth.LIS.core.LogiRec.LrFileHeadRead object at 0x10058e850>: "File header"
2012-02-08 17:43:45,103 WARNING LrTableRead(): Discarding duplicate row b'BS7 ' in table b'CONS'
0x000614b0 <TotalDepth.LIS.core.LogiRec.LrTableRead object at 0x10058e7d0>: "Well site data"
0x0006353e <TotalDepth.LIS.core.LogiRec.LrDFSRRead object at 0x10058e850>: "Data format specification record"
0x00065a44 <TotalDepth.LIS.core.LogiRec.LrFileTailRead object at 0x10058e850>: "File trailer"
0x00065a8e <TotalDepth.LIS.core.LogiRec.LrFileHeadRead object at 0x10058e7d0>: "File header"
2012-02-08 17:43:45,116 WARNING LrTableRead(): Discarding duplicate row b'BS7 ' in table b'CONS'
2012-02-08 17:43:45,124 WARNING LrTableRead.__init__(): Tell: 0x65ad8 LD index: 0x32 Error: FileRead.unpack(): Bytes: b'\x00' not enough for struct that needs: 12 bytes.
0x00065ad8 <TotalDepth.LIS.core.LogiRec.LrTableRead object at 0x10058e850>: "Well site data"
0x00067b7a <TotalDepth.LIS.core.LogiRec.LrDFSRRead object at 0x10058e7d0>: "Data format specification record"
0x000d2c44 <TotalDepth.LIS.core.LogiRec.LrFileTailRead object at 0x10058e7d0>: "File trailer"
CPU time = 0.064 (S)
Bye, bye!
ScanLogiData.py¶
Scans a LIS79 file and reports the Logical Record structure.
Arguments¶
One argument that will be treated as a path to a LIS file.
Options¶
| Option | Description |
|---|---|
| --version | Show program’s version number and exit |
| -h, --help | Show this help message and exit. |
| -k, --keep-going | Keep going as far as sensible. [default: False] |
| -d DUMP, --dump=DUMP | Dump complete data at these integer positions (ws separated, hex/dec). [default: ] |
| -l LOGLEVEL, --loglevel=LOGLEVEL | Log Level (debug=10, info=20, warning=30, error=40, critical=50) [default: 20] |
| -v, --verbose | Verbose output, this outputs a representation of table data and DFSRs. |
Examples¶
Example of scanning a LIS file:
$ python3 ScanLogiData.py LIS.lis
Cmd: ScanLogiData.py LIS.lis
Offset Length Type Logical Data
0x00000000 58 128 b'\x80\x00RUN1R .S01\x00\x00DAT2TF '...
0x0000003E 8190 34 b'"\x00IA\x04\x00TYPE CONS\x00A\x04\x00MNEM HI'...
0x00002060 300 64 b'@\x00\x01\x02O\x00\x00\x02\x02O\x00\x00\x03\x04I\x00\x00\x00\x18\x04\x02O\x00\x01\x08\x04D?N\x07_\t'...
0x00002190 1010 0 b"\x00\x00F@'\xde\xbe76\xfb@\xd6\x1a\xc0@\xd0\xdc\xc7D\xe0\xa6P\xba\x83\x18\x00F@&\xa6\xbe-"...
0x00002586 1010 0 b'\x00\x00E\xff\xe9S\xbe:\x1f\x82@\xfe%\xc9@\xf7\xd5\xb7EA\x90\xda\xba\x83\x18\x00E\xff\xe6\xe3\xbe\x8a'...
0x0000297C 1010 0 b'\x00\x00E\xff\x82\xea\xbe-\xe1\xa8@\xd83\xb6@\xf0\x0f\x0fET\x149D\xc8\x08\xc5E\xff\x80y\xbe-'...
0x00002D72 1010 0 b'\x00\x00E\xff\x1c\x80\xbd\xba\x7f\x19@\xc4\xbf\xe8@y\x0b\xb3E\xc0\x08\x03D\xd5\xednE\xff\x1a\x10\xbd\xb4'...
0x00003168 1010 0 b'\x00\x00E\xfe\xb6\x16\xbe\x12\xde\xf0@\xcbl\xe7@zF\xc2Ew\xba/D\xd0\xca\xd9E\xfe\xb3\xa6\xbe\x17'...
0x0000355E 1010 0 b'\x00\x00E\xfeO\xac\xbe40\x85@\xcc4\x8d@of\xd9E\xc1F\xd8D\xd4\xaa+E\xfeM<\xbe6'...
0x00003954 1010 0 b"\x00\x00E\xfd\xe9C\xbd\xb2\x19\xf0\xba\x83\x18\x00AK'%D\xed\x9d\xdbD\xd0\x17RE\xfd\xe6\xd3\xbd\xab"...
0x00003D4A 1010 0 b'\x00\x00E\xfd\x82\xd9\xba\x83\x18\x00\xba\x83\x18\x00\xba\x83\x18\x00\xba\x83\x18\x00D\xd0\xad\xf3E\xfd\x80i\xba\x83'...
0x00004140 938 0 b'\x00\x00E\xfd\x1co\xba\x83\x18\x00\xba\x83\x18\x00\xba\x83\x18\x00\xba\x83\x18\x00D\xd8\x8c\xb5E\xfd\x19\xff\xba\x83'...
0x000044EE 58 0 b'\x81\x00RUN1R .S01\x00\x00DAT2TF '...
Histogram of Logical Data lengths:
Bytes
58 [1] | +++++++++++
300 [1] | +++++++++++
938 [1] | +++++++++++
1010 [8] | ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
8190 [1] | +++++++++++
Histogram of Logical Record types:
0 [9] | ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
34 [1] | ++++++++++
64 [1] | ++++++++++
128 [1] | ++++++++++
CPU time = 0.001 (S)
Bye, bye!
First ScanLogiData.py echo’s the command line then it scans the file an writes out a table that has the following columns:
| Heading | Description |
|---|---|
| Offset | The file position of the start of the Physical Record as a hex integer. |
| Length | The length of the Logical Record as a decimal integer. |
| Type | The Logical Record type as a decimal integer. |
| Logical Data | The logical data payload. Only the first 32 bytes are shown. ... is shown if the payload is longer than 32 bytes. If the verbose or dump options are given then all bytes are shown. |
This is followed by an ASCII histogram of the lengths of all logical data with the following columns:
- The size in bytes.
- The frequency count.
- A series of + that is proportionate to the frequency count.
This is followed by an ASCII histogram of the lengths of all Logical Record types with the following columns:
- The size in bytes.
- The frequency count.
- A series of + that is proportionate to the frequency count.
Using the -d option expands the output when the file position value matches. So given the above then adding -d 0x44EE changes this:
...
0x000044EE 58 0 b'\x81\x00RUN1R .S01\x00\x00DAT2TF '...
...
To this:
...
0x000044EE 58 0 b'\x81\x00RUN1R .S01\x00\x00DAT2TF \x00 1024\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
...
Extracting Data from LIS¶
DumpFrameSet.py¶
Reads a LIS file and writes out tab separated values of each frame.
Arguments¶
- The path to the LIS file.
Options¶
| Option | Description |
|---|---|
| --version | Show program’s version number and exit |
| -h, --help | Show this help message and exit. |
| -k, --keep-going | Keep going as far as sensible. [default: False] |
| -l LOGLEVEL, --loglevel=LOGLEVEL | Log Level (debug=10, info=20, warning=30, error=40, critical=50) [default: 20] |
| -s, --summary | Display summary only [default: False]. |
Examples¶
$ python3 DumpFrameSet.py LIS.lis
Cmd: DumpFrameSet.py LIS.lis
2012-02-09 08:41:38,372 INFO Index.indexFile(): LIS.lis
<TotalDepth.LIS.core.LogPass.LogPass object at 0x101a0c510>
b'DEPT' [b'M '] b'SP ' [b'MV '] b'SN ' [b'OHMM'] b'ILD ' [b'OHMM'] b'CILD' [b'MMHO'] b'DT ' [b'US/M']
2052.98 -4.54908 1.34538 1.26348 386.599 -999.25
2052.83 -5.1372 1.36062 1.29521 500.511 -999.25
2052.68 -6.66747 1.38543 1.45786 595.623 -999.25
2052.53 -6.69616 1.43226 1.61085 592.447 -999.25
2052.37 -4.93782 1.51647 1.6622 590.846 -999.25
2052.22 -4.38823 1.66883 1.70584 586.092 -999.25
2052.07 -4.70347 1.8102 1.70607 577.873 -999.25
...
1996.44 -999.25 -999.25 -999.25 -999.25 -999.25
1996.29 -999.25 -999.25 -999.25 -999.25 -999.25
1996.14 -999.25 -999.25 -999.25 -999.25 -999.25
1995.99 -999.25 -999.25 -999.25 -999.25 -999.25
Sc Name Count Min Mean Max Std Dev. -- == ++ Bias Drift Activity
DEPT [M ] 375 2e+03 2.02e+03 2.05e+03 16.5 374 0 0 1 -0.152 0.000144
SP [MV ] 262 -13.7 -5.67 -0.769 2.66 124 0 137 -0.0498 0.0144 0.678
SN [OHMM] 252 0.866 1.36 1.98 0.277 123 0 128 -0.0199 -0.000719 0.0425
ILD [OHMM] 253 0.361 1.31 2.35 0.412 95 0 157 -0.246 0.00429 0.134
CILD [MMHO] 253 387 787 1.75e+03 236 130 0 122 0.0317 0.205 0.101
DT [US/M] 292 133 320 460 42.5 139 0 152 -0.0447 -0.451 0.106
CPU time = 0.047 (S)
Bye, bye!
The summary table at the end has the following columns:
| Heading | Description |
|---|---|
| Sc Name | The sub-channel name and units of measure. |
| Count | The number of non-null values. |
| Min | Minimum value. |
| Mean | Arithmetic mean of values. |
| Max | Maximum value. |
| Std Dev. | Standard deviation of values. |
| -- | Number of values that are a decrease over the previous value. |
| == | Number of values that are equal to the previous value. |
| ++ | Number of values that are an increase over the previous value. |
| Bias | (-- - ++) / total |
| Drift | (last value - first value) / number of values |
| Activity | The RMS exponent change. |
LisToHtml.py¶
Generates HTML from input LIS file or directory to an output destination.
Arguments¶
- The path to the input LIS file or directory.
- The path to the output file or directory, any directories will be created as necessary.
Options¶
| Option | Description |
|---|---|
| --version | Show program’s version number and exit |
| -h, --help | Show this help message and exit. |
| -g, --glob | File pattern match. [default none] |
| -j JOBS, --jobs=JOBS | Max processes when multiprocessing. Zero uses number of native CPUs [8]. -1 disables multiprocessing. [default: -1] |
| -k, --keep-going | Keep going as far as sensible. [default: False] |
| -l LOGLEVEL, --loglevel=LOGLEVEL | Log Level (debug=10, info=20, warning=30, error=40, critical=50) [default: 20] |
| -v, --verbose | Verbose output, this outputs a representation of table data and DFSRs. |
| -r, --recursive | Process input recursively. [default: False] |
Examples¶
Command to process a directory of LIS:
$ python3 LisToHtml.py Simple LIS_plot/Simple_00
Output:
Cmd: LisToHtml.py Simple LIS_plot/Simple_00
plotLogInfo:
FileInfo: "Simple/LIS.lis" -> "LIS_plot/Simple_00/LIS.lis.html" 17 (kb) LR count=4 t=0.070
FileInfo: "Simple/RW.lis" -> "LIS_plot/Simple_00/RW.lis.html" 843 (kb) LR count=12 t=3.206
FileInfo: "Simple/RW_No_TIF.lis" -> "LIS_plot/Simple_00/RW_No_TIF.lis.html" 833 (kb) LR count=12 t=3.200
CPU time = 6.568 (S)
Exec. time = 6.568 (S)
Bye, bye!
For each file the output lists:
- Input file.
- Output HTML file.
- File size.
- Count of Logical Records.
- Execution time.
In the output directory there will be an index.html file, for example:
The columns are:
- The name of the LIS file.
- The size of the LIS file.
- Count of Logical Records.
- Execution time.
- Processing rate.
In the linked HTML file is a summary of the content of the LIS file.
The Log Pass merits several entries, the first summarises the frame shape and the shape of each channel, for example:
Then there is a couple of tables, the first summarises the X axis and the second summarises each channel (min, max mean etc.), for example:
TableHistogram.py¶
Provides a count of elements in LIS tables.
Arguments¶
- A path to a LIS file or directory of LIS files.
Options¶
| Option | Description |
|---|---|
| --version | Show program’s version number and exit |
| -h, --help | Show this help message and exit. |
| -k, --keep-going | Keep going as far as sensible. [default: False] |
| -r, --recursive | Process input recursively. [default: False] |
| -s, --structure | Display table structure (row/col range). [default: False] |
| --type=LRTYPE | Logical record table type e.g. 34. [default: 34] |
| --name=NAME | Logical record table name e.g. PRES. [default: ] |
| --row=ROW | Logical record table row e.g. “GR ”. [default: ] |
| --col=COL | Logical record table column e.g. “LEDG”. [default: ] |
| -l LOGLEVEL, --loglevel=LOGLEVEL | Log Level (debug=10, info=20, warning=30, error=40, critical=50) [default: 20] |
Examples¶
Count of all entries regardless of the table/row/column that they appear in:
$ python3 TableHistogram.py -l 40 Simple/
Cmd: TableHistogram.py -l 40 Simple/
======================== Count of all table entries =======================
{"(34, b' ')": 1414,
"(34, b'0.445')": 5,
"(34, b'0.621')": 5,
"(34, b'013529700231')": 7,
"(34, b'1')": 5,
"(34, b'1.22')": 5,
"(34, b'1.70')": 2,
"(34, b'116')": 2,
"(34, b'12.25')": 2,
"(34, b'15')": 5,
"(34, b'15-4-76')": 5,
"(34, b'17')": 5,
"(34, b'17.5')": 5,
"(34, b'19')": 5,
"(34, b'1976')": 7,
"(34, b'2')": 2,
"(34, b'2055.0')": 2,
"(34, b'2071.2')": 4,
"(34, b'25')": 2,
"(34, b'25-6-76')": 2,
"(34, b'257.0')": 7,
...
"(34, b'WN ')": 7,
"(34, b'YEAR')": 7,
'(34,)': 443}
====================== Count of all table entries END =====================
CPU time = 0.205 (S)
Bye, bye!
The result is a dictionary that has the key as a pair (lr_type, cell_value) and the value as a count of the number of occurrences.
If the -s option is used then an additional summary is provided:
=============================== Row entries ===============================
{(34, b'CONS', b'APIN'): 7,
(34, b'CONS', b'BLI '): 7,
(34, b'CONS', b'BS1 '): 7,
(34, b'CONS', b'BS2 '): 7,
(34, b'CONS', b'BS3 '): 7,
...
(34, b'CONS', b'WN '): 7,
(34, b'CONS', b'YEAR'): 7}
============================= Row entries END =============================
============================== Column entries =============================
{(34, b'CONS', b'ALLO'): 707,
(34, b'CONS', b'MNEM'): 707,
(34, b'CONS', b'PUNI'): 707,
(34, b'CONS', b'TUNI'): 707,
(34, b'CONS', b'VALU'): 707}
============================ Column entries END ===========================
This are dictionaries that have the key as a tripple (lr_type, table_name, row_name) and (lr_type, table_name, column_name) respectively and the value as a count of the number of occurrences.
Filtering by Logical Record type, table name, row name and column name (note quoting of spaces):
$ python3 TableHistogram.py -l 40 --type=34 --name=CONS --row="WN " --col=VALU Simple/
Cmd: TableHistogram.py -l 40 --type=34 --name=CONS --row=WN --col=VALU Simple/
======================== Count of all table entries =======================
{"(34, b'CONS', b'WN ', b'VALU', b'B897 - 14')": 1,
"(34, b'CONS', b'WN ', b'VALU', b'DIEKSAND 111A')": 3,
"(34, b'CONS', b'WN ', b'VALU', b'VOELKERSEN AZ4')": 3}
====================== Count of all table entries END =====================
CPU time = 0.174 (S)
Bye, bye!
The result is a dictionary that has the key as a quadruple (lr_type, table_name, row_name, column_name, cell_value) and the value as a count of the number of occurrences.
LIS Developer Tools¶
DeTif.py¶
Takes an input LIS79 file and writes out a new one without TIF markers.
Arguments¶
- The path to the input LIS file.
- The path to the output LIS file, any directories will be created as necessary.
Options¶
| Option | Description |
|---|---|
| --version | Show program’s version number and exit |
| -h, --help | Show this help message and exit. |
| -n, --nervous | Nervous mode (do no harm). [default: False] |
| -l LOGLEVEL, --loglevel=LOGLEVEL | Log Level (debug=10, info=20, warning=30, error=40, critical=50) [default: 20] |
Examples¶
DeTIF with nervous mode just examines the file:
$ python3 DeTif.py -n RW.lis RW_No_TIF.lis
Cmd: DeTif.py -n RW.lis RW_No_TIF.lis
stripTif(): Tell: 0x00000000 Len: 0x00000000 TIF: TIF True >: 0x 0 0x 0 0x 0
stripTif(): Tell: 0x0000000c Len: 0x0000003e TIF: TIF True >: 0x 0 0x 0 0x 4a
stripTif(): Tell: 0x00000056 Len: 0x00000400 TIF: TIF True >: 0x 0 0x 0 0x 456
stripTif(): Tell: 0x00000462 Len: 0x00000400 TIF: TIF True >: 0x 0 0x 4a 0x 862
stripTif(): Tell: 0x0000086e Len: 0x00000400 TIF: TIF True >: 0x 0 0x 456 0x c6e
stripTif(): Tell: 0x00000c7a Len: 0x00000400 TIF: TIF True >: 0x 0 0x 862 0x 107a
stripTif(): Tell: 0x00001086 Len: 0x00000400 TIF: TIF True >: 0x 0 0x c6e 0x 1486
stripTif(): Tell: 0x00001492 Len: 0x00000400 TIF: TIF True >: 0x 0 0x 107a 0x 1892
stripTif(): Tell: 0x0000189e Len: 0x00000400 TIF: TIF True >: 0x 0 0x 1486 0x 1c9e
...
stripTif(): Tell: 0x000d2b4e Len: 0x000000f6 TIF: TIF True >: 0x 0 0x d2748 0x d2c44
stripTif(): Tell: 0x000d2c50 Len: 0x0000003e TIF: TIF True >: 0x 0 0x d2b42 0x d2c8e
2012-02-09 08:33:39,381 ERROR Premature EOF of input so output terminated: RawStream.readAndUnpack(): EOF; read b'' but need 12 bytes
CPU time = 0.022 (S)
Exec. time = 0.022 (S)
Bye, bye!
DeTIF with write:
$ python3 DeTif.py RW.lis RW_No_TIF.lis
Cmd: DeTif.py RW.lis RW_No_TIF.lis
2012-02-09 08:33:48,382 ERROR Premature EOF of input so output terminated: RawStream.readAndUnpack(): EOF; read b'' but need 12 bytes
CPU time = 0.019 (S)
Exec. time = 0.019 (S)
Bye, bye!
Index.py¶
This indexes a LIS file and prints out the result. It can also provide some performance measurements of the indexing operation. See Indexing LIS Files for more information about the design and performance of LIS indexing.
Arguments¶
- The path to a LIS file or a directory of LIS files.
Options¶
| Option | Description |
|---|---|
| --version | Show program’s version number and exit |
| -h, --help | Show this help message and exit. |
| -k, --keep-going | Keep going as far as sensible. [default: False] |
| -l LOGLEVEL, --loglevel=LOGLEVEL | Log Level (debug=10, info=20, warning=30, error=40, critical=50) [default: 20] |
| -j JOBS, --jobs=JOBS | Max processes when multiprocessing. Zero uses number of native CPUs [8]. -1 disables multiprocessing. [default: -1] |
| -t TIMES, --times=TIMES | Number of times to repeat the read [default: 1] |
| -s, --statistics | Dump timing statistics. [default: False] |
| -v, --verbose | Verbose output, this outputs a representation of table data and DFSRs. |
| -r, --recursive | Process input recursively. [default: False] |
Examples¶
Simple scan of a single file:
$ python3 Index.py Simple/LIS.lis
Cmd: Index.py Simple/LIS.lis
2012-02-09 09:36:28,039 INFO Index.indexFile(): Simple/LIS.lis
File size: 17708 (0.017 MB) Reference Time: 0.002459 (s) for Simple/LIS.lis pickleLen=4351 jsonLen=-1
Summary:
Results: 1
Errors: 0
Total: 1
CPU time = 0.004 (S)
Bye, bye!
Simple scan of a single file with verbose output:
$ python3 Index.py -v Simple/LIS.lis
Cmd: Index.py -v Simple/LIS.lis
2012-02-09 09:39:29,493 INFO Index.indexFile(): Simple/LIS.lis
<TotalDepth.LIS.core.FileIndexer.FileIndex object at 0x10197fdd0> "Simple/LIS.lis" [4]:
tell: 0x00000000 type=128 <TotalDepth.LIS.core.FileIndexer.IndexFileHead object at 0x10197fe10>
tell: 0x0000003e type= 34 name=b'CONS' <TotalDepth.LIS.core.FileIndexer.IndexTable object at 0x10197fe90>
<TotalDepth.LIS.core.LogPass.LogPass object at 0x101b071d0>
tell: 0x000044ee type=129 <TotalDepth.LIS.core.FileIndexer.IndexFileTail object at 0x101b07790>
=============================== All records ===============================
tell: 0x00000000 type=128 <TotalDepth.LIS.core.FileIndexer.IndexFileHead object at 0x10197fe10>
tell: 0x0000003e type= 34 name=b'CONS' <TotalDepth.LIS.core.FileIndexer.IndexTable object at 0x10197fe90>
<TotalDepth.LIS.core.LogPass.LogPass object at 0x101b071d0>
tell: 0x000044ee type=129 <TotalDepth.LIS.core.FileIndexer.IndexFileTail object at 0x101b07790>
============================= All records DONE ============================
================================ Log Passes ===============================
LogPass <TotalDepth.LIS.core.LogPass.LogPass object at 0x101b071d0>:
DFSR: <TotalDepth.LIS.core.LogiRec.LrDFSRRead object at 0x10197ff90>: "Data format specification record"
Frame plan: <TotalDepth.LIS.core.Type01Plan.FrameSetPlan object at 0x101b07210>: indr=0 frame length=24 channels=6
Channels: [b'DEPT', b'SP ', b'SN ', b'ILD ', b'CILD', b'DT ']
RLE: <TotalDepth.LIS.core.Rle.RLEType01 object at 0x101b07250>: func=None: [RLEItemType01: datum=8592 stride=1014 repeat=7 frames=42, RLEItemType01: datum=16704 stride=None repeat=0 frames=39]
X axis: first=2052.983 last=1995.986 frames=375 overall spacing=-0.1524 in optical units=b'M ' (actual units=b'M ')
Frame set: None
============================= Log Passes DONE =============================
=============================== Plot Records ==============================
============================ Plot Records DONE ============================
Min: 0.003 (s)
Max: 0.003 (s)
Mean: 0.003 (s)
File size: 17708 (0.017 MB) Reference Time: 0.002529 (s) for Simple/LIS.lis pickleLen=4351 jsonLen=-1
Summary:
Results: 1
Errors: 0
Total: 1
CPU time = 0.004 (S)
Bye, bye!
Scan of a directory (recursively) indexing each file 11 times and writing out statistics:
$ python3 Index.py -t11 -s -l 40 Simple/
Cmd: Index.py -t11 -s -l 40 ../../../../TDTestData/LIS/Simple
File size: 17708 (0.017 MB) Reference Time: 0.001670 (s) for Simple/LIS.lis pickleLen=4351 jsonLen=-1
File size: 863374 (0.823 MB) Reference Time: 0.043411 (s) for Simple/RW.lis pickleLen=18231 jsonLen=-1
File size: 853030 (0.814 MB) Reference Time: 0.039238 (s) for Simple/RW_No_TIF.lis pickleLen=18238 jsonLen=-1
Summary:
Size (kb) Time (s)
17.293 0.001670
843.139 0.043411
833.037 0.039238
Files: 3
Errors: 0
CPU time = 0.938 (S)
Bye, bye!
RandomFrameSetRead.py¶
For developers only. This is designed to measure the performance of loading and iterating across a frame-set.