Is there any way to speed up Mac systems to be faster

[blu3knight]

I have two systems here that I have done tests on.
System 1 is a Mac M3 Pro with 38 GB of Ram
System 2 is a Intel System with NVIDIA GeForce RTX 3070 (8GB GPU Ram), 64 GB Regular RAM

For regular inference and every other AI function the Mac runs circles around the Nvidia graphics card. But for xtts the mac is extremely slow.

With my calculations the following is true to achieve 1% of the workload.
Mac = 16 min 19 seconds per 1%
Nvidia = 2 min 35 seconds per 1%

With this being the case the mac will be at a 100% from start at 27 hours and 11 secons
The Nvidia system would achieve the same 100% in 4 hours and 19 minutes

The Nvidia system is 84% faster then the Mac.
Is there any improvements that can be done or is this just the the way it is and the performance will just be this slow.

[ROBERT-MCDOWELL]

did you select MPS as processor for your mac conversion?

[blu3knight]

Of course

[DrewThomasson]

Honestly until we find time (or someone creates a pull request 👀) that adds a TTS engine which has been optimized for MPS the performance isn't gonna be fantastic.

Being able to run on MPS and actually being optimized for MPS are two different things.

If you want faster processing times I would suggest using one of our smaller TTS engines in the meantime other than xtts

Here I made a wiki page on comparing TTS speeds for different engines on M1

https://github.com/DrewThomasson/ebook2audiobook/wiki/M1-Mac-CPU-speeds

[blu3knight]

Thank you for checking if you can patch the MPS in xtts. Shame to have a very powerful computer that is performing so bad because of lack of optimizaiton.

I am pretty technical and can try and test things (as an example both windows / Mac are all manual builds using Anaconda.) But not a developer, so I can vary and test things pretty easily.

[ROBERT-MCDOWELL]

yes I will need you to test the patch btw...

[blu3knight]

I will have to find a small Epub... the one I used for the above testing is only a few minutes and does not give the best results. Good for general voice tests. Not good for testing of the patch. If you have any recommendations please let me know and I will for now build the baseline using the current systems.

[ROBERT-MCDOWELL]

never mind the file you'll use. it's just to know if the MPS will be fully functionnal. btw for now I'm working on other issues and features so be patient I will contact you when it's time.

[blu3knight]

Thank you very much.

[blu3knight]

When I start off an ebook this warning message comes back (Mac M3Pro) does it mean anything

Warning
Using MPS - VRAM capacity could not be detected. -

[ROBERT-MCDOWELL]

MPS is used, but E2A cannot check how much VRAM your computer has... nothing serious

[ROBERT-MCDOWELL]

@blu3knight you could try on the coming update if it's better with MPS. I added some patch that can use full MPS.

[blu3knight]

Ok waiting for update but in the mean time I notice something strange when I ran some tests from command line.

Checking for MPS in the Python environment I get that MPS is running, but when I enable the following command.
python app.py --headless --ebook ~/Documents/Alita-Test.epub --fine_tuned ScarlettJohansson --device mps --tts_engine XTTSv2 > ~/Documents/2025-10-13-2215-out.txt

I get the following in the logs.
2025-10-13 22:16:02 INFO: Use device: cpu
2025-10-13 22:16:02 INFO: Loading: tokenize

Which is strange considering that it should be using MPS.

[ROBERT-MCDOWELL]

provide the full log.... something wrong with torch and your mac. torch does not recognize your mac GPU apparently

[blu3knight]

So just an FYI in the UI it shows MPS, no men when you activate it. I am going to troubleshoot a bit and try to figure it out. Also going to create another pull and use the built in installer vs the manual I created.

[blu3knight]

So just an FYI in the UI it shows MPS, no men when you activate it. I am going to troubleshoot a bit and try to figure it out. Also going to create another pull and use the built in installer vs the manual I created.

Adding some more info. Using the following script

import torch

if torch.backends.mps.is_available():
    print("MPS is available and PyTorch can use it.")
    mps_device = torch.device("mps")
    
    # Optional: Create a tensor on the MPS device to confirm functionality
    x = torch.ones(5, device=mps_device)
    print(f"Tensor created on MPS device: {x}")
else:
    print("MPS is not available or PyTorch is not configured to use it.")
    if not torch.backends.mps.is_built():
        print("PyTorch was not built with MPS support.")
    else:
        print("MPS not available because the current macOS version is not 12.3+ or you do not have an MPS-enabled device.")

Output is:

MPS is available and PyTorch can use it.
Tensor created on MPS device: tensor([1., 1., 1., 1., 1.], device='mps:0')

Attaching full log, something is strange in the log just pulling out some things....

v25.8.18 native mode
Using MPS - VRAM capacity could not be detected. -

Then when models are loaded....

2025-10-14 13:29:47 INFO: Loading these models for language: en (English):
=========================
| Processor | Package   |
-------------------------
| tokenize  | ewt       |
| ner       | ontonotes |
=========================

2025-10-14 13:29:47 INFO: Use device: cpu
2025-10-14 13:29:47 INFO: Loading: tokenize

[blu3knight]

Log
Output-2025-10-14.md

[ROBERT-MCDOWELL]

the question is are you using the OS python or the E2A virtual python env when you run this script?

[blu3knight]

I am using the python within Anaconda, and manual setup .... it is within the Virtual Environment I have set up, for both commands.

[blu3knight]

I am always working within the Virtual Environment set up with Anaconda.

python --version
Python 3.12.11

[ROBERT-MCDOWELL]

btw E2A xtts model uses already MPS... "Using MPS - VRAM capacity could not be detected. "
what you see using the CPU .is STANZA, a model that detect number and special math chars which is normal.
check at the start of the LOG

[blu3knight]

OK thank you for the reply.... That makes sense. My recommendation is some way of seprating this (or describing it) within the output of the log. When I write scripts for various things, I always try to separate with comments so confusion like this does not occure for people (like me) that do not know the depth of the code.

Also above when you mention that you patched the code, was it for ebook2audiobook github, or something else. As I need to understand what to watch to pull and update.

[ROBERT-MCDOWELL]

ok no worries.

[ccraig]

I have a M4 Max with 128GB RAM. I see no difference in performance between CPU and MPS, when selected. Takes about 3 min 30 sec, per 1%. Also, really cool project!

[ROBERT-MCDOWELL]

I have a M4 Max with 128GB RAM. I see no difference in performance between CPU and MPS, when selected. Takes about 3 min 30 sec, per 1%. Also, really cool project!

which version you're using?

[ROBERT-MCDOWELL]

try last git or last release

[ccraig]

Should I still use DEVICE_TAG=mps?

[ROBERT-MCDOWELL]

yes

DEVICE_TAG=mps ./ebook2audiobook.sh --script_mode build_docker this is for docker only.
if native,
just ./ebook2audiobook.sh

[ccraig]

Setting up now. I have a one page test ePub. Where will I find the install and first run logs? I'm running it native.

[ccraig]

Just ran the test.

MPS took 243 seconds
CPU took 123 seconds

[ROBERT-MCDOWELL]

ok so coqui-tts is still not faster with mps... once E2A will use torch 2.9.1 and coqui-tts v0.27.3 so it will be maybe better.
btw mps is part of cpu so use cpu for now.