Skip to content

Expand TTL to edges #1244

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Apr 22, 2025
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 47 additions & 15 deletions pages/querying/time-to-live.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,48 +7,50 @@ import { Callout } from 'nextra/components'

# Time to live (Enterprise)

Time-to-live allows a user to tag vertices with an expiration time. Once a vertex has expired, the vertex and all associated edges will be deleted.
Time-to-live allows a user to tag vertices and edges with an expiration time. Once a vertex has expired, the vertex and all its associated edges will be deleted.

<Callout type="warning">

The `TTL` label and `ttl` property are reserved names for TTL. See [Tagging vertices](#tagging-vertices) for more info.
The `TTL` label and `ttl` property are reserved names for TTL. See [Tagging objects](#tagging-objects) for more info.

</Callout>

<Callout type="info">

Time-to-live is implemented as a background job that periodically gets executed.
This is a best effort solution; meaning that even if a vertex expires, that does not mean it gets deleted right away, but eventually, once the background job gets executed.
Time-to-live is implemented as a background job that periodically gets executed.
This is a best effort solution; meaning that even if an object expires, that does not mean it gets deleted right away, but eventually, once the background job gets executed.

</Callout>

## Usage

In order to use the feature the user needs to:
1. [Configure TTL](#configuration)
2. [Tag vertices with an expiration time](#tagging-vertices)
2. [Tag vertices and edges with an expiration time](#tagging-objects)

Once that is done, a background job will periodically delete expired vertices and associated edges.
Once that is done, a background job will periodically delete expired vertices, its associated edges and expired edges.

### What is indexed

Time-to-live uses a label `TTL` and property `ttl` to tag vertices. A label+property value index is used to speed up query execution.
Edges are tagged using only the `ttl` property and are scanned using the global edge property index.

### Executed query

Time-to-live is implemented as a background job that execute the following query:
Time-to-live is implemented as a background job that execute the following queries:

```cypher
MATCH (n:TTL) WHERE n.ttl < $now WITH n LIMIT $batch DETACH DELETE n;
MATCH ()-[e]->() WHERE e.ttl < $now WITH e LIMIT $batch DELETE e;
```

The query DETACH DELETEs all vertices that have expired at this point in time.
The query DETACH DELETEs all vertices and DELETEs edges that have expired at this point in time.
The query is batched to limit the serialization errors and lost work that the error might cause.

<Callout type="warning">

Since time-to-live is implemented as a query like any other, the user might get serialization errors.
This can happen if the user is modifying an expired vertex.
This can happen if the user is modifying an expired vertex or edge.
The chance of serialization errors can be minimized by limiting the duration of write transactions.
In addition, the user can disable TTL before starting an important write transaction and re-enable it after commit. See [configuration](#configuration).

Expand Down Expand Up @@ -101,31 +103,61 @@ DISABLE TTL;

**NOTE:** Once disabled, ttl has to be re-enabled with a new configuration.

## Tagging vertices
## Tagging objects

In order to tag a vertex for expiration, the user needs to add the `TTL` label and `ttl` property.
Edges are tagged with only the `ttl` property. This allows the user to use TTL on any edge type.

`ttl` property defines when the vertex has expired as a number of microseconds since POSIX epoch.
POSIX epoch defined as starting from 1st of January 1970. Negative values define time before, while positive numbers the time since.
The user can simply input the number or can use builtin Memgraph functions to define the expiration time.

Arbitrary time since epoch:
Define arbitrary time since epoch when creating an object:
```cypher
CREATE (:TTL{ttl:123});
CREATE ()-[e:E{ttl:123}]->();
```

Mark an already existing object with an arbitrary time since epoch:
```cypher
MATCH (n) SET n:TTL, n.ttl=123;
MATCH ()-[e]->() SET e.ttl=123;
```

Set object expiration to time at creation:
```cypher
CREATE (:TTL{ttl:timestamp()});
CREATE ()-[e:A{ttl:timestamp()}]->();
```

Set vertex to expire now:
Update object's expiration to now:
```cypher
MATCH (n) SET n:TTL, n.ttl=timestamp();
MATCH ()-[e]->() SET e.ttl=timestamp();
```

Set vertex to expire 11 hours and 25 minutes from now:
Create an object that will expire 11 hours and 25 minutes after creation:
```cypher
CREATE(:TTL{ttl:timestamp() + timestamp(duration({hour:11, minute:25}))});
CREATE ()-[e:B{ttl:timestamp() + timestamp(duration({hour:11, minute:25}))}]->();
```

Update object's expiration to 11 hours and 25 minutes from now:
```cypher
MATCH (n) SET n:TTL, n.ttl=timestamp() + timestamp(duration({hour:11, minute:25}));
MATCH ()-[e]->() SET e.ttl=timestamp() + timestamp(duration({hour:11, minute:25}));
```

Create object that will expire on July 15th 2024 at 14:15:
```cypher
CREATE (:TTL{ttl:timestamp(LocalDateTime("2024-07-15T14:15:00"))});
CREATE ()-[e:C{ttl:timestamp(LocalDateTime("2024-07-15T14:15:00"))}]->();
```

Set vertex to expire on July 15th 2024 at 14:15:
Update object's so it expires on July 15th 2024 at 14:15:
```cypher
MATCH (n) SET n:TTL, n.ttl=timestamp(LocalDateTime("2024-07-15T14:15:00"));
MATCH ()-[e]->() SET e.ttl=timestamp(LocalDateTime("2024-07-15T14:15:00"));
```

## Authorization
Expand Down Expand Up @@ -153,7 +185,7 @@ List of features and their supported status:

Time-to-live configuration is tenant based; meaning that the feature will need to be enabled and configured manually for each tenant.

### Replication
### Replication

Time-to-live background job will be execute only on MAIN and the changes will be replicated.
While the TTL effect is replicated, the configuration is not. TTL needs to be configured manually on every instance that can become MAIN.
Expand Down