Model setup recipes

Building a Korean RAG index with Kakao's Kanana-2 3B embedding

It leads on Korean retrieval, and you can cut vectors to 128 dimensions.

We use the retrieval scores on the model card and the weight and vector math to pick a configuration that fits a Mac, then index documents with sentence-transformers.

Contents

Requirements and key details

The 3B model outputs 2,560 dimensions, the 1.3B model 1,280, and both accept 4,096 tokens at once.
MRL truncation to 1,024, 512, 256, or 128 dimensions cuts storage by up to twenty times.
It leads on Korean retrieval, while Qwen3-Embedding-4B scores higher on English MTEB.

An embedding model places questions and passages in one space

Kanana-2 embedding is a Korean and English retrieval model Kakao released on October 29, 2026. It turns each document passage and each user question into a numeric vector, then returns passages whose vectors sit closest to the question. The 3B model outputs 2,560 dimensions, the 1.3B model 1,280, and both accept up to 4,096 tokens¹ at once.

Kakao converted the Kanana-2 SLM to bidirectional attention² and pools the token outputs into one vector by averaging. It does not write the next token like a generative model; it reads the whole input once and returns a vector, which is what retrieval, RAG³, and similarity scoring need.

An illustration of one long document split into several smaller cards laid out on a desk — Split documents into passage cards, then return the cards closest to the question vector.

How high are the Korean retrieval scores?

The MTEB numbers were measured without any task prompt⁴, feeding raw text. On nano-BEIR (ko) the 3B model averages 0.6088 and the 1.3B model 0.5866; across the six Korean retrieval tasks (Ko-Retrieval) the 3B model averages 0.8211. In the same table Qwen3-Embedding-4B scored 0.5522 and 0.7837, and Qwen3-Embedding-0.6B scored 0.4845 and 0.7279.

English ranks differently. On MTEB (eng, v2), Qwen3-Embedding-4B scores 0.6661 against 0.6317 for the 3B Kanana model. Which model you pick depends on whether your corpus is mostly Korean or carries a large English share.

Scroll sideways to compare all columns.

MTEB scores from the model card (raw text, no task prompt, full dimensions)
Model	nano-BEIR(ko)	Ko-Retrieval	MTEB(eng v2)
kanana-2-3b-embedding	0.6088	0.8211	0.6317
kanana-2-1.3b-embedding	0.5866	0.7958	0.6197
Qwen3-Embedding-4B	0.5522	0.7837	0.6661
Qwen3-Embedding-0.6B	0.4845	0.7279	0.6311
EmbeddingGemma-300m	0.4564	0.7295	0.6247

Truncating to 128 dimensions cuts storage twentyfold

Matryoshka representation learning (MRL) trains the model so that a prefix of the vector still retrieves well. The 3B model can be truncated to 1,024, 512, 256, or 128 dimensions, and at 256 dimensions it still holds 0.5683 on nano-BEIR (ko) and 0.7873 on Ko-Retrieval.

Storage is the dimension count times four bytes. A 2,560-dimension vector costs 10,240 bytes per passage, so 100,000 passages take about 1.0 GB. At 256 dimensions the same passages take 102 MB, a tenth of that, and at 128 dimensions 51 MB, a twentieth. The same disk holds ten to twenty times as many passages.

Vector storage by passage count at float32 (text and index overhead excluded)
Dimensions	10k passages	100k passages
2,560	102MB	1.0GB
1,024	41MB	410MB
512	20MB	205MB
256	10MB	102MB
128	5MB	51MB

An illustration of document stacks and several cards on a desk, with arrows running from one card to another — Splitting passages below 4,096 tokens at index time keeps retrieval stable.

Indexing documents on a Mac

Load the model with sentence-transformers and pass 256 as truncate_dim to truncate in one step. On Apple Silicon, set device to mps to use the GPU⁵. The first run downloads the weights, so the 3B model needs roughly 6 GB of disk and memory.

Split passages so none exceeds 4,096 tokens, and use the same model and dimension for questions and passages. Rank by cosine similarity, then re-score the top candidates with a reranker if you need more precision. Record the model name and dimension in the index file so vectors from different runs never get mixed.

index.py

from sentence_transformers import SentenceTransformer

model = SentenceTransformer(
    "kakaocorp/kanana-2-3b-embedding",
    trust_remote_code=True,
    truncate_dim=256,
    device="mps",
)
vectors = model.encode(chunks, batch_size=8, normalize_embeddings=True)
print(vectors.shape)  # (passages, 256)

On a Mac set device to mps; change only truncate_dim to switch dimensions.

License and limits

The weights ship under the Kanana Open License. Running your own service is allowed, but reselling access through an API⁶, or selling the model as part of SI or on-device products, needs a separate commercial license from Kakao. Any redistribution must carry the "Powered by Kanana" notice and a copy of the license.

The 3B model is about 6 GB of bf16⁷ weights, which leaves room on a 16 GB Mac, but vectors and source text take more memory as the corpus grows. If English dominates your corpus, compare it with the higher-MTEB model on the same questions; if Korean document search is the main job, build an index with this model first and check whether the right passages come back on top.

Two computers compare document indexing and retrieval results — Use the same model and dimension for the index and the question to compare scores.

Terminology notes

Token — A unit into which a model divides input or output for processing.
Back to the text
Attention — A neural-network operation that calculates relationships between input tokens and combines information from each position into the current representation.
Back to the text
Retrieval-Augmented Generation — A method that retrieves material relevant to a query, adds it to the model input, and generates an answer from that context.
Back to the text
Prompt — The input text sent to a model. For generative models it describes the desired result and strongly shapes the output.
Back to the text
GPU — A processor designed to handle many calculations in parallel. It performs model computations during AI inference.
Back to the text
API — A defined interface that lets other code call a program’s functions.
Back to the text
BF16 — A 16-bit floating-point format for model values. Its exponent has the same width as FP32, giving it a similarly wide value range.
Back to the text

Frequently asked questions

What should you check first in this article?: We use the retrieval scores on the model card and the weight and vector math to pick a configuration that fits a Mac, then index documents with sentence-transformers.
How much memory does it need?: Load the model with sentence-transformers and pass 256 as truncate_dim to truncate in one step. On Apple Silicon, set device to mps to use the GPU. The first run downloads the weights, so the 3B model needs roughly 6 GB of disk and memory.