Request a feature: Have Gradio lazy-load TTS models & unload models when idle

[CD11b]

Describe the feature

The container seems to be utilizing 6 Gb of memory while idle. It'd be nice to have the option to have models be loaded only after uploading a file for processing.

Screenshot schema or code
Provide a maximum of elements to help us to understand and create this new feature

ps aux --sort=-%mem | head -10

USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 1.1 20.3 27278676 6697816 pts/0 SLsl+ Nov04 125:23 python app.py --script_mode full_docker
root 91 0.0 2.5 7875056 849624 pts/0 Sl+ Nov04 0:39 python app.py --script_mode full_docker
root 70166 0.0 0.0 8540 4560 pts/1 R+ 16:23 0:00 ps aux --sort=-%mem
root 70167 0.0 0.0 2916 1776 pts/1 S+ 16:23 0:00 head -10
root 70160 0.3 0.0 2580 1720 pts/1 Ss 16:23 0:00 /bin/sh

[ROBERT-MCDOWELL]

models are loaded in memory to faster multiple conversion and avoid to load them again and again.
in the case of multiple users it's not possible to flush the memory every time.
if you want to free your 6GB ram (depends which tts engine you use , yourtts use less than 512MB but bark 4GB min)
just quit ebook2audiobook.
btw, in the coming update models not used by any users in their active session are automatically flushed.

[ROBERT-MCDOWELL]

memory management has been widely optimized since now.