|
--- |
|
base_model: Writer/Palmyra-Fin-70B-32K |
|
language: |
|
- en |
|
license: other |
|
license_name: writer-open-model-license |
|
license_link: https://writer.com/legal/open-model-license/ |
|
pipeline_tag: text-generation |
|
tags: |
|
- instruct |
|
- pytorch |
|
- finance |
|
- stock market |
|
- candlesticks |
|
- FinGPT |
|
- option trading |
|
- future stock prediction |
|
- trends prediction |
|
- Enterprise LLM |
|
- Enterprise |
|
- Enterprise ready |
|
- Banks |
|
- Wealth Management |
|
quantized_by: bartowski |
|
extra_gated_prompt: By clicking "Agree", you agree to the [License Agreement](https://writer.com/legal/open-model-license/) |
|
and acknowledge Writer's [Privacy Policy](https://writer.com/legal/acceptable-use/). |
|
extra_gated_fields: |
|
Name: text |
|
Email: text |
|
Organization or Affiliation: text |
|
Receive email updates and promotions on Writer products, services, and research?: |
|
type: select |
|
options: |
|
- 'Yes' |
|
- 'No' |
|
I acknowledge that this model is for non-commercial use only unless I acquire a separate license from Writer: checkbox |
|
widget: |
|
- example_title: Palmyra-Fin-70B-32k |
|
messages: |
|
- role: system |
|
content: You are an expert language model specialized in the financial services |
|
domain. Your responses should be detailed, accurate, and tailored to the needs |
|
of financial professionals. Consider ethical implications, long-term impacts, |
|
and current global financial trends in your advice. |
|
- role: user |
|
content: As the CFO of a technology company, we are considering issuing new equity |
|
to fund our expansion into international markets. Could you provide a brief |
|
analysis of the advantages and disadvantages of issuing new equity, especially |
|
in the context of maintaining our current share price and investor relations? |
|
output: |
|
text: 'As the CFO of a technology company, considering issuing new equity to fund |
|
expansion into international markets is a strategic decision that requires a |
|
thorough analysis of its advantages and disadvantages. Issuing new equity can |
|
be an effective way to raise capital, but its crucial to weigh the benefits |
|
against the potential dilution of existing shareholders ownership and its impact |
|
on your share price. Heres a balanced view of the key advantages and disadvantages |
|
to help you make an informed decision:\n\nAdvantages:\n\n1. **Capital Raising**: |
|
Issuing new equity is an effective way to raise capital for expansion into international |
|
markets. This capital can be used to establish a presence in new markets, hire |
|
local teams, and develop marketing campaigns tailored to those regions.\n\n2. |
|
**Diversified Shareholder Base**: Issuing new equity can attract new investors, |
|
potentially including international investors. This can help broaden your shareholder |
|
base, which may improve liquidity and provide greater market stability.\n\n3. |
|
**Enhanced Credibility**: Issuing new equity can be perceived by the market |
|
as a vote of confidence in your company''s growth prospects, enhancing your |
|
credibility with existing and potential investors.\n\n4. **Strategic Partnerships**: Issuing |
|
new equity can facilitate strategic partnerships or collaborations. For example, |
|
issuing shares to a strategic partner in a new market can help you establish |
|
a local presence and gain access to their customer base.\n\n5. **Employee Incentives**: Issuing |
|
new equity can be used to incentivize employees, especially when vesting is |
|
tied to specific performance metrics or time-based vesting schedules. This can |
|
help attract and retain top talent.\n\nDisadvantages:\n\n1. **Dilution of Ownership**: Issuing |
|
new equity dilutes the ownership percentage of existing shareholders. This dilution |
|
can cause a decrease in the share price of your company''s stock.\n\n2. **Dilution |
|
of Earnings**: Issuing new equity dilutes earnings per share, which could cause |
|
a decrease in your company''s share price.\n\n3. **Increased Shareholder Scrutiny**: |
|
Issuing new equity may attract new shareholders who will demand greater transparency |
|
and scrutiny of your operations. This could increase legal and financial reporting |
|
expenses, make some activities more time-consuming, and may also place undue |
|
strain on your personnel.\n\n4. **Issuance Costs and Expenses**: Issuing new |
|
equity involves significant legal, accounting, and other expenses. These costs |
|
could include, among other things, registration fees, filing fees, printing |
|
fees, and the compensation of any underwriters. The total amount of these costs |
|
may be substantial.\n\n5. **Market Volatility**: Issuing new equity can cause |
|
market volatility in the price of your company''s stock. The sale of these |
|
shares, or the perception that sales could occur, could cause the price of your |
|
company''s stock to fall. This could also make it more difficult for you to |
|
sell equity or debt securities in the future at a time or price that you deem |
|
reasonable or appropriate.\n\n6. **Perception of Weakness**: The sale of additional |
|
equity securities may lead to a perception by the market that your company''s |
|
management believes its existing capital resources will be insufficient for |
|
its operations. This perception could cause your company''s stock price to decline.\n\n7. |
|
**Restrictions on Dividends**: If you are incorporated in a state that limits |
|
your ability to pay dividends on your common stock while you have preferred |
|
stock outstanding, then the rights of the holders of your common stock may be |
|
subject to limitations on the declaration of dividends.\n\n8. **Anti-Takeover |
|
Effects**: Issuing new equity may have anti-takeover effects. For example, if |
|
your company has a staggered board of directors, shares underlying new equity |
|
could be issued by your board of directors in one or more rights plans or similar |
|
plans. The rights will entitle all shareholders of record on a certain date |
|
to purchase shares of preferred stock upon the occurrence of certain events, |
|
such as a person acquiring a certain percentage of your companys outstanding |
|
voting stock. The rights will expire prior to the commencement of any business |
|
combination with a person who beneficially owns 20% or more of your companys |
|
outstanding voting stock. The rights may have the effect of entrenching your |
|
companys management team and may deprive you of the opportunity to sell your |
|
company to potential acquirers at a premium over prevailing prices. This potential |
|
inability to obtain a control premium could reduce the price of your company''s |
|
stock.\n\n9. **Impact on Employee Stock Options**: Issuing new equity may cause |
|
your companys employee stock options to become less valuable. This could adversely |
|
affect your companys ability to attract and retain employees.\n\n10. **Alternative |
|
Financing Options**: Depending on your companys stage of development and market |
|
conditions, it may be possible to obtain needed capital through alternative |
|
financing sources, such as a second public offering, a private placement of |
|
securities, or loans from banks or related or unrelated parties. These financing |
|
sources may have terms favorable to your company compared to those obtained |
|
through an issuance of new equity.\n\nIn the context of maintaining your current |
|
share price and investor relations, it''s essential to consider the following:\n\n1. |
|
**Communicate Effectively**: It is crucial to communicate the reasons behind |
|
issuing new equity clearly and transparently to your existing shareholders. |
|
This can help manage expectations and reduce potential negative perceptions.\n\n2. |
|
**Share Price Impact**: Analyze the potential dilution of earnings per share |
|
and the ownership percentage of existing shareholders. Consider the impact of |
|
issuing new equity on your companys share price in the short and long term.\n\n3. |
|
**Investor Relations**: Engage with your existing shareholders to understand |
|
their concerns and address them proactively. This can help maintain a positive |
|
relationship with your investor base.\n\n4. **Equity Incentive Plans**: Ensure |
|
that your equity incentive plans are aligned with your companys long-term goals |
|
and that vesting schedules are structured to incentivize employees appropriately.\n\n5. |
|
**Alternative Financing Options**: Carefully evaluate alternative financing |
|
options and consider the potential impact of each on your share price and investor |
|
relations.\n\nIn conclusion, issuing new equity to fund expansion into international |
|
markets can be a viable option, but it is essential to carefully weigh the advantages |
|
and disadvantages and consider alternative financing options. Effective communication |
|
with your existing shareholders is critical to managing expectations and maintaining |
|
a positive relationship with your investor base.' |
|
model-index: |
|
- name: Palmyra-Fin-70B-32k |
|
results: [] |
|
--- |
|
|
|
## Llamacpp imatrix Quantizations of Palmyra-Fin-70B-32K |
|
|
|
Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b3496">b3496</a> for quantization. |
|
|
|
Original model: https://huggingface.co/Writer/Palmyra-Fin-70B-32K |
|
|
|
All quants made using imatrix option with dataset from [here](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8) |
|
|
|
Run them in [LM Studio](https://lmstudio.ai/) |
|
|
|
## Prompt format |
|
|
|
``` |
|
<|begin_of_text|><|start_header_id|>system<|end_header_id|> |
|
|
|
{system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|> |
|
|
|
{prompt}<|eot_id|><|start_header_id|>assistant<|end_header_id|> |
|
|
|
|
|
``` |
|
|
|
## Download a file (not the whole branch) from below: |
|
|
|
| Filename | Quant type | File Size | Split | Description | |
|
| -------- | ---------- | --------- | ----- | ----------- | |
|
| [Palmyra-Fin-70B-32K-Q8_0.gguf](https://huggingface.co/bartowski/Palmyra-Fin-70B-32K-GGUF/tree/main/Palmyra-Fin-70B-32K-Q8_0) | Q8_0 | 74.98GB | true | Extremely high quality, generally unneeded but max available quant. | |
|
| [Palmyra-Fin-70B-32K-Q6_K.gguf](https://huggingface.co/bartowski/Palmyra-Fin-70B-32K-GGUF/tree/main/Palmyra-Fin-70B-32K-Q6_K) | Q6_K | 57.89GB | true | Very high quality, near perfect, *recommended*. | |
|
| [Palmyra-Fin-70B-32K-Q5_K_M.gguf](https://huggingface.co/bartowski/Palmyra-Fin-70B-32K-GGUF/tree/main/Palmyra-Fin-70B-32K-Q5_K_M) | Q5_K_M | 49.95GB | true | High quality, *recommended*. | |
|
| [Palmyra-Fin-70B-32K-Q5_K_S.gguf](https://huggingface.co/bartowski/Palmyra-Fin-70B-32K-GGUF/blob/main/Palmyra-Fin-70B-32K-Q5_K_S.gguf) | Q5_K_S | 48.66GB | false | High quality, *recommended*. | |
|
| [Palmyra-Fin-70B-32K-Q4_K_L.gguf](https://huggingface.co/bartowski/Palmyra-Fin-70B-32K-GGUF/blob/main/Palmyra-Fin-70B-32K-Q4_K_L.gguf) | Q4_K_L | 43.30GB | false | Uses Q8_0 for embed and output weights. Good quality, *recommended*. | |
|
| [Palmyra-Fin-70B-32K-Q4_K_M.gguf](https://huggingface.co/bartowski/Palmyra-Fin-70B-32K-GGUF/blob/main/Palmyra-Fin-70B-32K-Q4_K_M.gguf) | Q4_K_M | 42.52GB | false | Good quality, default size for must use cases, *recommended*. | |
|
| [Palmyra-Fin-70B-32K-Q4_K_S.gguf](https://huggingface.co/bartowski/Palmyra-Fin-70B-32K-GGUF/blob/main/Palmyra-Fin-70B-32K-Q4_K_S.gguf) | Q4_K_S | 40.35GB | false | Slightly lower quality with more space savings, *recommended*. | |
|
| [Palmyra-Fin-70B-32K-Q3_K_XL.gguf](https://huggingface.co/bartowski/Palmyra-Fin-70B-32K-GGUF/blob/main/Palmyra-Fin-70B-32K-Q3_K_XL.gguf) | Q3_K_XL | 38.06GB | false | Uses Q8_0 for embed and output weights. Lower quality but usable, good for low RAM availability. | |
|
| [Palmyra-Fin-70B-32K-IQ4_XS.gguf](https://huggingface.co/bartowski/Palmyra-Fin-70B-32K-GGUF/blob/main/Palmyra-Fin-70B-32K-IQ4_XS.gguf) | IQ4_XS | 37.90GB | false | Decent quality, smaller than Q4_K_S with similar performance, *recommended*. | |
|
| [Palmyra-Fin-70B-32K-Q3_K_L.gguf](https://huggingface.co/bartowski/Palmyra-Fin-70B-32K-GGUF/blob/main/Palmyra-Fin-70B-32K-Q3_K_L.gguf) | Q3_K_L | 37.14GB | false | Lower quality but usable, good for low RAM availability. | |
|
| [Palmyra-Fin-70B-32K-Q3_K_M.gguf](https://huggingface.co/bartowski/Palmyra-Fin-70B-32K-GGUF/blob/main/Palmyra-Fin-70B-32K-Q3_K_M.gguf) | Q3_K_M | 34.27GB | false | Low quality. | |
|
| [Palmyra-Fin-70B-32K-IQ3_M.gguf](https://huggingface.co/bartowski/Palmyra-Fin-70B-32K-GGUF/blob/main/Palmyra-Fin-70B-32K-IQ3_M.gguf) | IQ3_M | 31.94GB | false | Medium-low quality, new method with decent performance comparable to Q3_K_M. | |
|
| [Palmyra-Fin-70B-32K-Q3_K_S.gguf](https://huggingface.co/bartowski/Palmyra-Fin-70B-32K-GGUF/blob/main/Palmyra-Fin-70B-32K-Q3_K_S.gguf) | Q3_K_S | 30.91GB | false | Low quality, not recommended. | |
|
| [Palmyra-Fin-70B-32K-IQ3_XS.gguf](https://huggingface.co/bartowski/Palmyra-Fin-70B-32K-GGUF/blob/main/Palmyra-Fin-70B-32K-IQ3_XS.gguf) | IQ3_XS | 29.31GB | false | Lower quality, new method with decent performance, slightly better than Q3_K_S. | |
|
| [Palmyra-Fin-70B-32K-Q2_K_L.gguf](https://huggingface.co/bartowski/Palmyra-Fin-70B-32K-GGUF/blob/main/Palmyra-Fin-70B-32K-Q2_K_L.gguf) | Q2_K_L | 27.40GB | false | Uses Q8_0 for embed and output weights. Very low quality but surprisingly usable. | |
|
| [Palmyra-Fin-70B-32K-Q2_K.gguf](https://huggingface.co/bartowski/Palmyra-Fin-70B-32K-GGUF/blob/main/Palmyra-Fin-70B-32K-Q2_K.gguf) | Q2_K | 26.38GB | false | Very low quality but surprisingly usable. | |
|
| [Palmyra-Fin-70B-32K-IQ2_M.gguf](https://huggingface.co/bartowski/Palmyra-Fin-70B-32K-GGUF/blob/main/Palmyra-Fin-70B-32K-IQ2_M.gguf) | IQ2_M | 24.12GB | false | Relatively low quality, uses SOTA techniques to be surprisingly usable. | |
|
| [Palmyra-Fin-70B-32K-IQ2_S.gguf](https://huggingface.co/bartowski/Palmyra-Fin-70B-32K-GGUF/blob/main/Palmyra-Fin-70B-32K-IQ2_S.gguf) | IQ2_S | 22.24GB | false | Low quality, uses SOTA techniques to be usable. | |
|
| [Palmyra-Fin-70B-32K-IQ2_XXS.gguf](https://huggingface.co/bartowski/Palmyra-Fin-70B-32K-GGUF/blob/main/Palmyra-Fin-70B-32K-IQ2_XXS.gguf) | IQ2_XXS | 19.10GB | false | Very low quality, uses SOTA techniques to be usable. | |
|
|
|
## Embed/output weights |
|
|
|
Some of these quants (Q3_K_XL, Q4_K_L etc) are the standard quantization method with the embeddings and output weights quantized to Q8_0 instead of what they would normally default to. |
|
|
|
Some say that this improves the quality, others don't notice any difference. If you use these models PLEASE COMMENT with your findings. I would like feedback that these are actually used and useful so I don't keep uploading quants no one is using. |
|
|
|
Thanks! |
|
|
|
## Credits |
|
|
|
Thank you kalomaze and Dampf for assistance in creating the imatrix calibration dataset |
|
|
|
Thank you ZeroWw for the inspiration to experiment with embed/output |
|
|
|
## Downloading using huggingface-cli |
|
|
|
First, make sure you have hugginface-cli installed: |
|
|
|
``` |
|
pip install -U "huggingface_hub[cli]" |
|
``` |
|
|
|
Then, you can target the specific file you want: |
|
|
|
``` |
|
huggingface-cli download bartowski/Palmyra-Fin-70B-32K-GGUF --include "Palmyra-Fin-70B-32K-Q4_K_M.gguf" --local-dir ./ |
|
``` |
|
|
|
If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run: |
|
|
|
``` |
|
huggingface-cli download bartowski/Palmyra-Fin-70B-32K-GGUF --include "Palmyra-Fin-70B-32K-Q8_0/*" --local-dir ./ |
|
``` |
|
|
|
You can either specify a new local-dir (Palmyra-Fin-70B-32K-Q8_0) or download them all in place (./) |
|
|
|
## Which file should I choose? |
|
|
|
A great write up with charts showing various performances is provided by Artefact2 [here](https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9) |
|
|
|
The first thing to figure out is how big a model you can run. To do this, you'll need to figure out how much RAM and/or VRAM you have. |
|
|
|
If you want your model running as FAST as possible, you'll want to fit the whole thing on your GPU's VRAM. Aim for a quant with a file size 1-2GB smaller than your GPU's total VRAM. |
|
|
|
If you want the absolute maximum quality, add both your system RAM and your GPU's VRAM together, then similarly grab a quant with a file size 1-2GB Smaller than that total. |
|
|
|
Next, you'll need to decide if you want to use an 'I-quant' or a 'K-quant'. |
|
|
|
If you don't want to think too much, grab one of the K-quants. These are in format 'QX_K_X', like Q5_K_M. |
|
|
|
If you want to get more into the weeds, you can check out this extremely useful feature chart: |
|
|
|
[llama.cpp feature matrix](https://github.com/ggerganov/llama.cpp/wiki/Feature-matrix) |
|
|
|
But basically, if you're aiming for below Q4, and you're running cuBLAS (Nvidia) or rocBLAS (AMD), you should look towards the I-quants. These are in format IQX_X, like IQ3_M. These are newer and offer better performance for their size. |
|
|
|
These I-quants can also be used on CPU and Apple Metal, but will be slower than their K-quant equivalent, so speed vs performance is a tradeoff you'll have to decide. |
|
|
|
The I-quants are *not* compatible with Vulcan, which is also AMD, so if you have an AMD card double check if you're using the rocBLAS build or the Vulcan build. At the time of writing this, LM Studio has a preview with ROCm support, and other inference engines have specific builds for ROCm. |
|
|
|
Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski |
|
|
|
|
