Humminbird® File Structure
Introduction
PING-Mapper is a new and optimized version of PyHum [1] [2]. Since the release of PyHum, additional and enhanced functionality has been identified by the software authors and end-users, including Python 3 compatibility. This can only be achieved with a complete understanding of the Humminbird® recording file structure. This report documents new findings on the file structure of Humminbird® sonar recordings, essential for processing and exporting raw sonar data.
SD Card Files
Sonar recordings from Humminbird® side imaging sonar systems are saved to a SD card inserted into the control head. Each sonar recording consists of a DAT file and commonly named subdirectory containing SON and IDX files. The directory of saved recordings have the following structure:
Rec00001.DAT
├── Rec00001
│ ├── B001.IDX
│ ├── B001.SON
│ ├── B002.IDX
│ ├── B002.SON
│ ├── B003.IDX
│ ├── B003.SON
│ ├── B004.IDX
│ └── B004.IDX
Rec00002.DAT
├── Rec00002
│ ├── B001.IDX
│ ├── B001.SON
│ ├── B002.IDX
│ ├── B002.SON
│ ├── B003.IDX
│ ├── B003.SON
│ ├── B004.IDX
│ └── B004.IDX
....
DAT File Structure
The DAT file contains metadata that applies to the sonar recording. It includes information related water type specified on the sonar unit, the Unix date and time when the sonar recording began, geographic location where the recording began, name of the recording, number of sonar records, and length of the recording. The size (in bytes) of the DAT file varies by Humminbird® model (and potentially firmware). The following sections indicate the offset from start of the DAT file and description of the data for each model.
9xx/ 11xx/ Helix/ Solix Series
The DAT file structure is the same for the 9xx, 11xx, Helix, and Solix series models.
TABLE: DAT Structure
| Name | Offset (9xx, 11xx, Helix) | Offset (Solix) | Description |
|---|---|---|---|
| DAT Beginning | +0 | +0 | Beginning of DAT File |
| Water Type | +1 | +1 | 0=’fresh’ (freshwater); 1=’deep salt’; 2=’shallow salt’; otherwise=’unknown’ |
| - | +2-3 | +2-3 | - |
| Firmware Version (?) | +4 | +4 | Installed firmware version (?) |
| - | +8-19 | +8-19 | - |
| Unix Date/Time | +20 | +20 | Recording start date and time |
| UTM X | +24 | +24 | EPSG 3395 Easting |
| UTM Y | +28 | +28 | EPSG 3395 Northing |
| Recording Name | +32 | +32 | Recording name |
| - | +42-43 | +42-43 | Unknown |
| Number of Records | +44 | +44 | Number of pings |
| Length | +48 | +48 | Length (in milliseconds) of sonar recording |
| - | +52-59 | +52-91 | - |
| DAT End | +60 | +92 | End of DAT File |
Onix Series
The Onix series has a different structure from other Humminbird® models. The first 24 bytes are in binary containing information about water type, number of pings in the recording, total time of recording, and ping size in bytes. Following the binary header are ascii strings (human readable) containing additional information, with each piece of information encapsulated with <attribute=value>.
TABLE: Binary Header
| Name | Offset | Description |
|---|---|---|
| DAT Beginning | +0 | Beginning of DAT File |
| Water Type | +1 | 0=’fresh’ (freshwater); 1=’deep salt’; 2=’shallow salt’; otherwise=’unknown’ |
| Unknown | +2-3 | - |
| Number of Pings | +4 | Number of sonar records/pings |
| Length | +8 | Length (in milliseconds) of sonar recording |
| Ping Size (Bytes) | +12 | Number of returns per ping |
| First Ping Period | +16 | First ping period (in milliseconds) |
| Beam Mask | +20 | Unknown |
| Spacer | +24 | Spacer preceding ascii text |
TABLE: Ascii Text
| Name | Example |
|---|---|
| Version | < Version=SonarRecordThumbVersion > |
| Number of Pings | < NumberOfPings=11712 > |
| Total Time Ms | < TotalTimeMs=143092 > |
| Ping Size Bytes | < PingSizeBytes=1652 > |
| First Ping Period | < FirstPingPeriodMs=1 > |
| Beam Mask | < BeamMask=30 > |
| Chirp 1 Start Freq | < Chirp1StartFrequency=0 > |
| Chirp 1 End Freq | < Chirp1EndFrequency=0 > |
| Chirp 2 Start Freq | < Chirp2StartFrequency=0 > |
| Chirp 2 End Freq | < Chirp2EndFrequency=0 > |
| Chirp 3 Start Freq | < Chirp3StartFrequency=0 > |
| Chirp 3 End Freq | < Chirp3EndFrequency=0 > |
| Source Device Model ID 2D | < SourceDeviceModelId2D=1001 > |
| Source Device Model ID SI | < SourceDeviceModelIdSI=1001 > |
| Source Device Model ID DI | < SourceDeviceModelIdDI=1001 > |
IDX and SON File Structure
A SON file contains every sonar ping for a specific sonar channel (see table below) while the IDX file stores the byte offset and time ellapsed for each sonar ping. The IDX file allows quick navigation to locate pings in the SON file but can become corrupt due to power failure during the survey. Decoding the SON file without the IDX file requires additional information, outlined in the sections below.
TABLE: Sonar Channel File Names
| File Name | Description | Frequency |
|---|---|---|
| B000.SON | Down Scan Low Frequency | 50/83 kHz |
| B001.SON | Down Scan High Frequency | 200 kHz |
| B002.SON | Side Scan Port | 455/800/1,200 kHz |
| B003.SON | Side Scan Starboard | 455/800/1,200 kHz |
| B004.SON | Down Scan MEGA Frequency | 1,200 kHz |
Each SON file contains all the pings (ping header and returns) that were recorded. Each ping begins with a header, containing metadata specific to that ping (see Header Structure below). The header is followed by 8-bit (0-255 Integer) values representing the returns for that ping. All data stored in SON files are signed integer big endian.
Ping Structure
The number of bytes for a ping varies in two ways. First, the number of bytes corresponding to ping attributes vary by model (and potentially firmware version), resulting in varying header length. Second, the number of ping returns vary depending on the range set while recording the sonar. The variability in the size of a ping across recordings and Humminbird® models make automatic decoding of the file a non-trivial task. Consistent structure between recordings and Humminbird® models, however, has been identified.
Each ping begins with the same four hexidecimal values: C0 DE AB 21. This sequence is common to all sonar recordings encountered to date. The header then terminates with the following hexidecimal sequence: A0 ** ** ** ** 21 where the ** ** ** ** is a 32-byte unsigned integer indicating the number of sonar returns that are recorded immediately after 21. By counting the number of bytes beginning at C0 and terminating at 21, the correct header length can be determined. Three different header lengths have been identified:
TABLE: Header Length by Model
| Header Length | Humminbird Model |
|---|---|
| 67 Bytes | 9xx |
| 72 Bytes | 11xx, Helix, Onix |
| 152 Bytes | Solix |
Header Structure
The header for a ping contains attributes specific to that ping. Information about the ping location, time elapsed since beginning of the recording, heading, speed, depth, etc. are contained in this structure. The attribute is preceded by a hexidecimal value that is unique for the data that follows, referred to as a tag. For example, Depth is tagged by a hexidecimal value of 87. While the variety of information stored in the header varies by Humminbird® model, tags consistently identify the type of information that follows. The following sections indicate the tags, the attribute that follows the tag, and byte offset for the attribute by model.
TABLE: Ping Header Structure
| Name | Description | Hex Tag | 9xx | 11xx, Helix, Onix | Solix |
|---|---|---|---|---|---|
| Ping #1 | Beginning of ping | C0 | +0 | +0 | +0 |
| Header Start | Beginning of ping header | 21 | +3 | +3 | +3 |
| Record Number | Unique ping ID | 80 | +5 | +5 | +5 |
| Time Elapsed | Time elapsed (msec) | 81 | +10 | +10 | +10 |
| UTM X | EPSG 3395 easting coord. | 82 | +15 | +15 | +15 |
| UTM Y | EPSG 3395 northing coord. | 83 | +20 | +20 | +20 |
| Heading Quality | Quality flag1 | 84 | +25 | +25 | +25 |
| Heading | Vessel heading (1/10 deg) | - | +27 | +27 | +27 |
| Speed Quality | Quality flag1 | 85 | +30 | +30 | +30 |
| Speed | Vessel speed (cm/sec) | - | +32 | +32 | +32 |
| NA | Unknown data contents | 86 | - | +35 | +35 |
| Depth | Sonar depth (cm) | 87 | +35 | +40 | +40 |
| NA | Unknown data contents | - | - | - | +44-83 |
| Sonar Beam | Sonar beam ID2 | 50 | +40 | +45 | +85 |
| Voltage Scale | Voltage scale (1/10 volt) | 51 | +42 | +47 | +87 |
| Frequency | Sonar beam frequency (Hz) | 92 | +44 | +49 | +89 |
| NA | Unknown data contents | - | +48-60 | +53-65 | +89-145 |
| Return Count | Number ping returns (n) | A0 | +62 | +67 | +147 |
| Header End | End of ping header | 21 | +66 | +72 | +152 |
| Ping Returns | Sonar intensity [0-255] | 21 | +67 | +73 | +152 |
| Ping #2 | Beginning of ping | C0 | +67+n+1 | +73+n+1 | +152+n+1 |
| …3 | … | … | … | … | … |
References
[1] Buscombe, D., Grams, P. E., & Smith, S. M. C. (2015). Automated Riverbed Sediment Classification Using Low-Cost Sidescan Sonar. Journal of Hydraulic Engineering, 142(2), 06015019. https://doi.org/10.1061/(ASCE)HY.1943-7900.0001079
[2] Buscombe, D. (2017). Shallow water benthic imaging and substrate characterization using recreational-grade sidescan-sonar. Environmental Modelling and Software, 89, 1–18. https://doi.org/10.1016/j.envsoft.2016.12.003
-
See Sonar Channel File Names table for frequency values. ↩
-
Pattern repeats for duration of sonar recording. ↩