> For the complete documentation index, see [llms.txt](https://nccr-liri.gitbook.io/annotation-web-interface-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://nccr-liri.gitbook.io/annotation-web-interface-docs/technical/models-window.md).

# Models Window

This component takes care of the display of the WhisperSeg models. User can select models to annotate the audio or train new custom models.

### Get labels from WhisperSeg API

```jsx
const callWhisperSeg = async (event) => {
    event.preventDefault()

    passWhisperSegIsLoadingToTrack(true)
    const path = import.meta.env.VITE_BACKEND_SERVICE_ADDRESS+'get-labels'

    const annotatedAreas = filterAndConvertAnnotatedAreasForWhisper()
    const convertedLabels = filterAndConvertLabelsForWhisper()

    const requestParameters = {
        audio_id: audioId,
        annotated_areas: annotatedAreas,
        human_labels: convertedLabels,
        model_name: selectedInferenceModel,
        min_frequency: minFreqInference,
        token: authToken,
        nfft: nfft,
        sampling_rate: globalSamplingRate,
    }

    try {
        const response = await axios.post(path, requestParameters)
        const whisperObjects = response.data.labels

        // Create new species, Individuals and Clusternames in the Species panel from the whisper labels
        const updatedSpeciesArray = createSpeciesFromImportedLabels(whisperObjects, speciesArray)
        passSpeciesArrayToApp(updatedSpeciesArray)

        // Assign current config to all labels created by Whisperseg
        let whisperLabels = assignCurrentConfigToLabelsWithoutIt(whisperObjects)

        // Assign Species Information to the new labels
        whisperLabels = assignSpeciesInformationToImportedLabels(updatedSpeciesArray, whisperObjects)

        const annotatedAreaLabels = labels.filter( label => label.species === ANNOTATED_AREA)
        const combinedLabels = whisperLabels.concat(annotatedAreaLabels)
        passLabelsToTrack(combinedLabels)
        passShowModelsWindowToWhisperSeg(false)
    } catch (error){
        if (error.response.status === 403){
            toast.error('Access to WhisperSeg denied due to incorrect access token.')
        } else {
            toast.error('Something went wrong with your request. Check the console to view the error.')
            console.error(error)
        }
    } finally {
        passWhisperSegIsLoadingToTrack(false)
    }
}
```

`callWhisperSeg()` makes a request to the WhisperSeg API to generate labels. Before we can make that request we will have to convert our custom label objects into generic objects using `filterAndConvertLabelsForWhisper()`. In addition, we will filter out the annotated areas from our `labels` array and pass them to the WhisperSeg API separately, in addition to some other properties (see `requestParameters` variable).  \
\
Once the WhisperSeg API has returned a response we first update `speciesArray` with the newly created species, individuals and clusternames. We then convert the generic objects into our custom label objects using `assignSpeciesInformationToImportedLabels()`. As a last step remove the annotated areas from the `labels` array and combine them with the new `whisperLabels` array, which then becomes the new value of `labels`. 

### Train a custom model

```jsx
const handleClickSubmitTrainingRequestBtn = async (event) => {
    event.preventDefault()

    let annotatedAreas = filterAndConvertAnnotatedAreasForWhisper()
    const convertedLabels = filterAndConvertLabelsForWhisper()

    const allModels = [...modelsAvailableForInference, ...modelsAvailableForFinetuning, ...modelsCurrentlyTrained]

    for (const model of allModels){
        if (model.model_name === newModelName) {
            toast.error(`Model with the name "${newModelName}" already exists.`)
            return
        }
    }

    if (!convertedLabels.length){
        toast.error('You must provide at least one label to train the model on.')
        return
    }
    
    if (!annotatedAreas.length){
        toast.info('Provide at least one annotated Area.')
        return
    }

    const path= import.meta.env.VITE_BACKEND_SERVICE_ADDRESS+'finetune-whisperseg'

    const requestParameters = {
        audio_id: audioId,
        annotated_areas: annotatedAreas,
        human_labels: convertedLabels,
        new_model_name: newModelName,
        initial_model_name: selectedFinetuningModel,
        min_frequency: minFreqFinetune,
        token: authToken,
        nfft: nfft,
        sampling_rate: globalSamplingRate,
        ignore_cluster: ignoreCluster? 1:0
    }

    try {
        await axios.post(path, requestParameters)
        toast.success('Custom model started training and will be available soon.')
        const updatedArray = [...currentlyTrainedModelsNames, newModelName]
        passCurrentlyTrainedModelsNamesToWhisperSeg(updatedArray)
        setNewModelName('')

    } catch (error){
        if (error.response.status === 403){
            toast.error('Access to WhisperSeg denied due to incorrect access token.')
        } else {
            toast.error('Something went wrong with your request. Check the console to view the error.')
            console.error(error)
        }
    }
}
```

`handleClickSubmitTrainingRequestBtn()` sends a training request for a custom model to the WhisperSeg API. Before we can make that request we will have to convert our custom label objects into generic objects using `filterAndConvertLabelsForWhisper()`. In addition, we will filter out the annotated areas from our `labels` array and pass them to the WhisperSeg API separately, in addition to some other properties (see `requestParameters` variable).  \
\
Three checks will be made before that: The user will have to provide at least one annotated area and one label, as well as a new unique model name that follows a naming scheme.