Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG]克隆声音效果很不好 #357

Open
lonngxiang opened this issue Jul 7, 2024 · 4 comments
Open

[BUG]克隆声音效果很不好 #357

lonngxiang opened this issue Jul 7, 2024 · 4 comments
Labels
bug Something isn't working

Comments

@lonngxiang
Copy link

用的docker部署,

docker run -it --gpus all -v /ai/fish-speech/checkpoints:/exp/checkpoints  -p 7899:7860 -p 8022:8000 lengyue233/fish-speech bas

python -m tools.api --listen 0.0.0.0:8000 --llama-checkpoint-path "checkpoints/fish-speech-1.2" --decoder-checkpoint-path "checkpoints/fish-speech-1.2/firefly-gan-vq-fsq-4x1024-42hz-generator.pth" --decoder-config-name firefly_gan_vq

image

请求案例:

import requests
import io

def generate_speech(text, output_file="output.wav"):
    url = "http://192.168.2.238:8022/v1/invoke"  # 替换为实际的API地址
    
    payload = {
        "text": text,
        "format": "wav",
        "streaming": False,
        "reference_audio": "checkpoints/aa.wav",
        "reference_text": "你说的对, 但是原神是一款由米哈游自主研发的开放世界手游.",

    }
    
    response = requests.post(url, json=payload)
    
    if response.status_code == 200:
        with open(output_file, "wb") as f:
            f.write(response.content)
        print(f"音频已保存到 {output_file}")
    else:
        print(f"请求失败,状态码:{response.status_code}")
        print(response.text)

# 使用示例
text_to_speak = "这组池塘生物的摄影作品由本迪克特·普莱尔在德国巴伐利亚的基希贝格拍摄。作品中展示了多种微小的水生生物,包括水螅、团藻、水蚤、螺旋虫和合尾藻等。这些生物在自然环境中通常难以被肉眼观察到,但通过暗视野技术和高倍率的物镜放大,它们被清晰地捕捉并呈现出来。"

generate_speech(text_to_speak)

生成的结果,短的是残酷音频,output.wav是生成结果,效果非常不好
output.zip

@lonngxiang lonngxiang added the bug Something isn't working label Jul 7, 2024
@lonngxiang
Copy link
Author

还有个缺陷,请求参考音频文件只能和服务器一个地址下才可以正常推理

@PoTaTo-Mika
Copy link
Contributor

请参照文档中的内容运行项目,我们不保证按照其它步骤操作能够得到与样例一致的效果。

@lonngxiang
Copy link
Author

lonngxiang commented Jul 7, 2024

我是参考文档运行的这个api 服务
image

@leng-yue
Copy link
Member

我们最近几天会放出 sft 模型, 以及一些自动 rerank 的算法

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants