Track

The track component is where audio is visualized as a spectrogram and waveform and where the user annotates the actual audio. Each track represents one channel of the uploaded audio file, e.g. uploading a stereo file will create two new tracks. Each track also contains a side-bar with user controls and a so called labels canvas, to display the labels, as well as an individuals canvas to display the individuals. Let's have a look at some key variables.

// General
const [audioId, setAudioId] = useState(trackData.audioID)
const [canvasWidth, setCanvasWidth] = useState(window.innerWidth - 200)
const resizeTimeoutRef = useRef(null)

audioId is a unique identifier for the audio channel inside this track. It is generated and returned by the backend.

canvasWidth stores the current width of the following canvases: label canvas, spectrogram canvas, waveform canvas, timeaxis canvas and overview canvas.

resizeTimeoutRef stores a timeout object, which used in a useEffect hook that recalculates the current value of canvasWidth in intervals of 200ms, when the user changes the window size.

// Spectrogram
const specCanvasRef = useRef(null)
const specImgData = useRef(null)
const [spectrogram, setSpectrogram] = useState(trackData.spectrogram)

specCanvasRef is the reference object to tie the spectrogram canvas into the DOM.

specImgData is another reference object that tracks the content of that canvas (think of it as a screenshot). This value will used for functions that refresh the canvas many times in a row but the underlying spectrogram remains unchanged (e.g. dragging or hovering labels), instead of making a request to the backend to compute a new spectrogram each time.

spectrogram tracks the state of the most recent returned spectrogram from the backend. This value is updated any time a spectrogram is recomputed by the backend (e.g. zoom, scroll and audio upload events).

// Frequency
const frequenciesCanvasRef = useRef(null)
const [frequencies, setFrequencies] = useState(trackData.frequencies)
const [showFrequencyLines, setShowFrequencyLines] = useState(false)
const [frequencyLines, setFrequencyLines] = useState({maxFreqY: 0, minFreqY: SPEC_CVS_HEIGHT})
let draggedFrequencyLinesObject = null

frequenciesCanvasRef is the reference object to tie the frequencies canvas into the DOM.

frequencies tracks the state of the most recent returned frequencies array from the backend. This value is updated any time a new frequency Array is returned by the backend (e.g. when the user changes the frequency range or uploads new audio).

showFrequencyLines is a boolean that, set to true, toggles on the display of the min- and max-frequency lines in the spectrogram canvas.

frequencyLines is an object that keeps track of the Y position of both both frequency lines (this is a pixel value, not the actual frequency).

draggedFrequencyLinesObject is a copy of frequencyLines it used to update the values in real-time when dragging the frequency lines, instead of setting the state of frequencyLines asynchronously.

// Labels and Individuals Canvases
const [showLabelAndIndividualsCanvas, setShowLabelAndIndividualsCanvas] = useState(true)  

showLabelAndIndividualsCanvas is a boolean that, set to true, toggles on the display of the individuals and label canvas.

// Label Canvas
const labelCanvasRef = useRef(null)

labelCanvasRef is the reference object to tie the label canvas into the DOM.

// Individuals Canvas
const individualsCanvasRef = useRef(null)
const numberOfIndividuals = speciesArray.reduce((total, speciesObj) => total + speciesObj.individuals.length, 0)
const [showLabelAndIndividualsCanvas, setShowLabelAndIndividualsCanvas] = useState(true)

individualsCanvasRef is the reference object to tie the individuals canvas into the DOM.

numberOfIndividuals is an integer that is used to calculate the height of the individuals canvas.

// Time Axis and Overview Container
const overviewTimeAxisContainerRef = useRef(null)

overviewTimeAxisContainerRef is the reference object to tie the container of those canvases into the DOM.

// Time Axis
const timeAxisRef = useRef(null)

timeAxisRef is the reference object to tie the timeaxis canvas into the DOM.

// Overview Window
const overviewRef = useRef(null)
let newViewportStartFrame = null
let newViewportEndFrame = null
let widthBetween_xStartTime_mouseX = null
let widthBetween_xEndTime_mouseX = null

overviewAxisRef is the reference object to tie the overview canvas into the DOM.

newViewportStartFrame and newViewportEndFrame store the start- and end-frame of the viewport bar in seconds and are updated in real-time upon drag events by the user to allow for a smooth animation.

widthBetween_xStartTime_mouseX and widthBetween_xEndTime_mouseX store the the width in pixels and are used for real-time calculation of the overview bars position when the user drags it.

// Labels
const [labels, setLabels] = useState([])
const [activeLabel, setActiveLabel] = useState(null)
let draggedActiveLabel = null
let clickedLabel = undefined
let lastHoveredLabel = {labelObject: null, isHighlighted: false}

labels is an array that stores all label objects of this track.

activeLabel is the object that tracks which label is currently selected. It is used in other tracks to extend the onset and offset of the active label to those sibling components.

draggedActiveLabel is a copy of activeLabel that is being updated in real-time when the user drags the onset or offset of the active label, to allow a smooth animation.

clickedLabel is a copy of the currently selected label that is being updated in real-time when the user drags the onset or offset of that label, to allow a smooth animation.

lastHoveredLabel is an object that stores the currently hovered label, which is being rendered separately to the labels that are not being hovered

// Audio
const playheadRef = useRef(new Playhead(0))
const [audioSnippet, setAudioSnippet] = useState(null)
const [playWindowTimes, setPlayWindowTimes] = useState(null)

playheadRef is a reference object that stores an instance of the Playhead class. This custom class is used to update the playhead when playing the audio in real-time for a smooth animation.

audioSnippet stores the sound file of the audio visible in the current track viewport, when the user requests to playback the audio.

playWindowTimes is an object that stores the start- and end-time of the audio snippet.

// Waveform
const waveformCanvasRef = useRef(null)
const waveformImgData = useRef(null)
const [audioArray, setAudioArray] = useState(null)
const [waveformScale, setWaveformScale] = useState(25)
const [showWaveform, setShowWaveform] =  useState(true)

waveformCanvasRef is the reference object to tie the waveform canvas into the DOM.

waveformImgData is another reference object that tracks the content of that canvas (think of it as a screenshot). This value will used for functions that refresh the canvas many times in a row but the underlying waveform remains unchanged (e.g. dragging or hovering labels), instead of making a request to the backend to compute a new waveform each time.

audioArray is an array of data points returned by the backend used to draw the waveform.

waveformScale is a value to set the waveforms height (in both directions).

showWaveform is a boolean that, set to true, toggles on the display of the waveform canvas.

// File Upload
const [spectrogramIsLoading, setSpectrogramIsLoading] = useState(false)

spectrogramIsLoading is a boolean that conditionally renders various animations, buttons and elements on the page while the spectrogram is being computed by the backend.

// Local Parameters
const [showLocalConfigWindow, setShowLocalConfigWindow] = useState(false)
const [specCalMethod, setSpecCalMethod] = useState(trackData.specCalMethod ? trackData.specCalMethod : 'log-mel')
const [nfft, setNfft] = useState(trackData.nfft ? trackData.nfft : '')
const [binsPerOctave, setBinsPerOctave] = useState(trackData.binsPerOctave ? trackData.binsPerOctave: '')
const [minFreq, setMinFreq] = useState(trackData.minFreq ? trackData.minFreq : '')
const [maxFreq, setMaxFreq] = useState(trackData.maxFreq ? trackData.maxFreq : '')

showLocalConfigWindow is a boolean that, set to true, toggles on the display of the Local Configurations Window.

specCalMethod, nfft, binsPerOctave, minFreq and maxFreq track the current state of their corresponding input fields and radio buttons inside the local config window.

// WhisperSeg
const [whisperSegIsLoading, setWhisperSegIsLoading] = useState(false)

whisperSegIsLoading is a boolean that conditionally renders a loading bar while WhisperSeg is annotating the audio.

// Active Species
const activeSpecies = speciesArray.find(speciesObj =>
    speciesObj.individuals.some(individual => individual.isActive)
)

activeSpecies keeps track of the currently selected species in the species menu. Active means that any labels being made while be attributed to this species.

// Label Window
const [expandedLabel, setExpandedLabel] = useState(null)
const [globalMouseCoordinates, setGlobalMouseCoordinates] = useState(null)

expandedLabel keeps track of the current label expanded in the label window.

globalMouseCoordinates keep track of the users mouse position when he clicked on a label to expand it. These coordinates are being used to render the label window at this mouse position.

// Icons
const activeIcon = showWaveform ? icon : iconSmall
const activeIconBtnStyle = showWaveform ? iconBtn : iconBtnSmall

activeIcon keeps track of the currently active Icon style. Regular icons when the waveform is rendered, small icons when it's hidden, so all icons fit into the now smaller track sidebar.

activeIconBtnStyle keeps track of the currently active Icon button style. Regular button styles when the waveform is rendered, small button styles when it's hidden, so all buttons fit into the now smaller track sidebar.

// Scroll Context
const { setAnyWindowsOpen } = useOpenWindowsContext()

setAnyWindowsOpen updates the OpenWindowsContext object in OpenWindowsContext.jsx. This value is used to keep track of open windows inside Track.jsx and temporarily disable the arrow key scroll events in App.jsx, so that the user can use the arrow keys inside the open windows input fields without triggering a scroll call.