# Models Window

This component takes care of the display of the WhisperSeg models. User can select models to annotate the audio or train new custom models.

### Get labels from WhisperSeg API

```jsx
const callWhisperSeg = async (event) => {
    event.preventDefault()

    passWhisperSegIsLoadingToTrack(true)
    const path = import.meta.env.VITE_BACKEND_SERVICE_ADDRESS+'get-labels'

    const annotatedAreas = filterAndConvertAnnotatedAreasForWhisper()
    const convertedLabels = filterAndConvertLabelsForWhisper()

    const requestParameters = {
        audio_id: audioId,
        annotated_areas: annotatedAreas,
        human_labels: convertedLabels,
        model_name: selectedInferenceModel,
        min_frequency: minFreqInference,
        token: tokenInference
    }

    try {
        const response = await axios.post(path, requestParameters)
        const whisperObjects = response.data.labels

        // Create new species, Individuals and Clusternames in the Species panel from the whisper labels
        const updatedSpeciesArray = createSpeciesFromImportedLabels(whisperObjects, speciesArray)
        passSpeciesArrayToApp(updatedSpeciesArray)

        // Assign Species Information to the new labels
        const whisperLabels = assignSpeciesInformationToImportedLabels(updatedSpeciesArray, whisperObjects)

        const annotatedAreaLabels = labels.filter( label => label.species === ANNOTATED_AREA)
        const combinedLabels = whisperLabels.concat(annotatedAreaLabels)
        passLabelsToTrack(combinedLabels)
        passShowModelsWindowToWhisperSeg(false)
    } catch (error){
        if (error.response.status === 403){
            toast.error('Access to WhisperSeg denied due to incorrect access token.')
        } else {
            toast.error('Something went wrong with your request. Check the console to view the error.')
            console.error(error)
        }
    } finally {
        passWhisperSegIsLoadingToTrack(false)
    }
}
```

`callWhisperSeg()` makes a request to the WhisperSeg API to generate labels. Before we can make that request we will have to convert our custom label objects into generic objects using `filterAndConvertLabelsForWhisper()`. In addition, we will filter out the annotated areas from our `labels` array and pass them to the WhisperSeg API separately, in addition to some other properties (see `requestParameters` variable).  \
\
Once the WhisperSeg API has returned a response we first update `speciesArray` with the newly created species, individuals and clusternames. We then convert the generic objects into our custom label objects using `assignSpeciesInformationToImportedLabels()`. As a last step remove the annotated areas from the `labels` array and combine them with the new `whisperLabels` array, which then becomes the new value of `labels`. 

### Train a custom model

```jsx
const handleClickSubmitTrainingRequestBtn = async (event) => {
    event.preventDefault()

    const annotatedAreas = filterAndConvertAnnotatedAreasForWhisper()
    const convertedLabels = filterAndConvertLabelsForWhisper()

    const allModels = [...modelsAvailableForInference, ...modelsAvailableForFinetuning, ...modelsCurrentlyTrained]

    for (const model of allModels){
        if (model.model_name === newModelName) {
            toast.error(`Model with the name "${newModelName}" already exists.`)
            return
        }
    }

    if (!annotatedAreas.length || !convertedLabels.length){
        toast.error('Provide at least one annotated Area and one label to train the new model on.')
        return
    }

    const path= import.meta.env.VITE_BACKEND_SERVICE_ADDRESS+'finetune-whisperseg'

    const requestParameters = {
        audio_id: audioId,
        annotated_areas: annotatedAreas,
        human_labels: convertedLabels,
        new_model_name: newModelName,
        initial_model_name: selectedFinetuningModel,
        min_frequency: minFreqFinetune,
        token: tokenFinetune
    }

    try {
        await axios.post(path, requestParameters)
        toast.success('Custom model started training and will be available soon.')
        const updatedArray = [...currentlyTrainedModelsNames, newModelName]
        passCurrentlyTrainedModelsNamesToWhisperSeg(updatedArray)
        setNewModelName('')

    } catch (error){
        if (error.response.status === 403){
            toast.error('Access to WhisperSeg denied due to incorrect access token.')
        } else {
            toast.error('Something went wrong with your request. Check the console to view the error.')
            console.error(error)
        }
    }
}
```

`handleClickSubmitTrainingRequestBtn()` sends a training request for a custom model to the WhisperSeg API. Before we can make that request we will have to convert our custom label objects into generic objects using `filterAndConvertLabelsForWhisper()`. In addition, we will filter out the annotated areas from our `labels` array and pass them to the WhisperSeg API separately, in addition to some other properties (see `requestParameters` variable).  \
\
Two checks will be made before that, the user will have to provide at least one annotated area and one label, as well as a new unique model name that follows a naming scheme.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://nccr-liri.gitbook.io/annotation-web-interface-docs/technical/models-window.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.