Add support for GGUF Phi-3

[dabs9]

Model description

Model is Phi-3 saved as a GGUF.

Open source status

• The model implementation is available
• The model weights are available

Provide useful links for the implementation

It would be the analog of this recent PR but for Phi 3: e462843#diff-55e3debce4586d986257d1b1acb48c8708504f4b3b63e413e89f27ac2cc738b3

[amyeroberts]

cc @SunMarc

[SunMarc]

Hey @dabs9, thanks for showing interest in GGUF integration with Phi-3 model ! I will try to tackle this when I get the time but I highly encourage you to do the integration yourself if you have the time ! As you saw from the Qwen integration, this should be fairly straightforward. I will mark it as a first good issue in case a community member is interested.

[a8nova]

Hello @SunMarc @amyeroberts - if nobody started on this, i would like to contribute to this by adding the GGUF for phi3, is following Qwen integration as a guide good enough to get started?

[SunMarc]

Hi @a8nova, feel free to add the gguf path for phi3 ! Following Qwen integration should be good enough indeed ! I'll try to help if you need any help ! If the integration is harder than expected, we might write a guide on how to add gguf path. Let us know how it goes ! Qwen PR here.

[a8nova]

Perfect, I created an initial PR: #31844. There is an 11 year old programmer that I am mentoring, he knows some python and I think this will be a great first issue for him, of course i will guide him and I will get help from you guys on things that are not clear, thanks!