Full-Text Search (Autocomplete) With SurrealDB In Less Than 10 Minutes

Sharing what we're hacking while building https://onelife.city

AI generated image for New York

It’s super quick and easy to setup full-text search with SurrealDB. And the icing on the cake is SurrealDB lets you customize your full-text search experience for various use cases via different tokenizers and filters. In this post, we’ll explore setting up full-text search for a very common use case aka autocomplete. I would highly recommend reading the following three articles from SurrealDB’s official documentation after you’ve gone through this post.

1. DEFINE ANALYZER

2. DEFINE INDEX

3. Search functions

Step 1: Create a few sample records in the entrepreneur table.

CREATE entrepreneur SET name = 'Elon Musk';
CREATE entrepreneur SET name = 'Peter Thiel';
CREATE entrepreneur SET name = 'Steve Jobs';
CREATE entrepreneur SET name = 'Bill Gates';
CREATE entrepreneur SET name = 'Bhavish Aggarwal';
CREATE entrepreneur SET name = 'Sam Altman';
CREATE entrepreneur SET name = 'Travis Kalanick';

Step 2: Create an analyzer named full_name using tokenizers blank and filters lowercase and edgengram. The blank tokenizer will split entrepreneur names into first names and last names based on whitespaces. The lowercase filter will let us perform case-insensitive searches. The edgengram filter will let us perform autocomplete searches based on partial text.

DEFINE ANALYZER full_name TOKENIZERS blank FILTERS lowercase, edgengram(1, 10);

Step 3: Create an index named entrepreneur_name on column name of the entrepreneur table which uses the analyzer full_name we created in step 2.

DEFINE INDEX entrepreneur_name ON TABLE entrepreneur COLUMNS name SEARCH ANALYZER full_name BM25;

Step 4: Play!

Searching for uppercase ‘S’:

SELECT
    id,
    name,
    search::score(1) AS score
FROM
    entrepreneur
WHERE
    name @1@ 'S'
ORDER BY
    score DESC

returns both, Steve Jobs and Sam Altman:

[
    {
        "id": "entrepreneur:7w6dtrzgrtr1b0f6a291",
        "name": "Steve Jobs",
        "score": 0.8395084738731384
    },
    {
        "id": "entrepreneur:h5qvj9fub3jr1s1c03ao",
        "name": "Sam Altman",
        "score": 0.8395084738731384
    }
]

Searching for lowercase ‘st’:

SELECT
    id,
    name,
    search::score(1) AS score
FROM
    entrepreneur
WHERE
    name @1@ 'st'
ORDER BY
    score DESC

returns only Steve Jobs:

[
    {
        "id": "entrepreneur:7w6dtrzgrtr1b0f6a291",
        "name": "Steve Jobs",
        "score": 0.8395084738731384
    },
]

Searching for lowercase ‘a’:

SELECT
    id,
    name,
    search::score(1) AS score
FROM
    entrepreneur
WHERE
    name @1@ 'a'
ORDER BY
    score DESC

returns both, Sam Altman and Bhavish Aggarwal:

[
    {
        "id": "entrepreneur:h5qvj9fub3jr1s1c03ao",
        "name": "Sam Altman",
        "score": 0.8395084738731384
    },
    {
        "id": "entrepreneur:cxbl2k1zwb7gll2wd3cw",
        "name": "Bhavish Aggarwal",
        "score": 0.6731036305427551
    }
]

Searching for uppercase ‘AGG’:

SELECT
    id,
    name,
    search::score(1) AS score
FROM
    entrepreneur
WHERE
    name @1@ 'AGG'
ORDER BY
    score DESC

returns only Bhavish Aggarwal:

[
    {
        "id": "entrepreneur:cxbl2k1zwb7gll2wd3cw",
        "name": "Bhavish Aggarwal",
        "score": 3.1767184734344482
    }
]

If you found this post useful, don’t forget to subscribe, more coming!