|
| 1 | +--- |
| 2 | +categories: |
| 3 | +- docs |
| 4 | +- develop |
| 5 | +- stack |
| 6 | +- oss |
| 7 | +- rs |
| 8 | +- rc |
| 9 | +- oss |
| 10 | +- kubernetes |
| 11 | +- clients |
| 12 | +description: Learn how to use Redis pipelines and transactions |
| 13 | +linkTitle: Pipelines/transactions |
| 14 | +title: Pipelines and transactions |
| 15 | +weight: 2 |
| 16 | +--- |
| 17 | + |
| 18 | +Redis lets you send a sequence of commands to the server together in a batch. |
| 19 | +There are two types of batch that you can use: |
| 20 | + |
| 21 | +- **Pipelines** avoid network and processing overhead by sending several commands |
| 22 | + to the server together in a single communication. The server then sends back |
| 23 | + a single communication with all the responses. This typically improves |
| 24 | + performance compared to sending the commands separately. See the |
| 25 | + [Pipelining]({{< relref "/develop/use/pipelining" >}}) page for more |
| 26 | + information. |
| 27 | +- **Transactions** guarantee that all the included commands will execute |
| 28 | + to completion without being interrupted by commands from other clients. |
| 29 | + See the [Transactions]({{< relref "/develop/interact/transactions" >}}) |
| 30 | + page for more information. |
| 31 | + |
| 32 | +## Execute a pipeline |
| 33 | + |
| 34 | +To execute commands in a pipeline, you first create a pipeline object |
| 35 | +and then add commands to it using methods that resemble the standard |
| 36 | +command methods (for example, `set()` and `get()`). The commands are |
| 37 | +buffered in the pipeline and only execute when you call the `execute()` |
| 38 | +method on the pipeline object. This method returns a list that contains |
| 39 | +the results from all the commands in order. |
| 40 | + |
| 41 | +Note that the command methods for a pipeline always return the original |
| 42 | +pipeline object, so you can "chain" several commands together, as the |
| 43 | +example below shows: |
| 44 | + |
| 45 | +<!-- Tested examples will replace the inline ones when they are approved. --> |
| 46 | +<!-- |
| 47 | +{{< clients-example pipe_trans_tutorial basic_pipe Python >}} |
| 48 | +{{< /clients-example >}} |
| 49 | +--> |
| 50 | +```python |
| 51 | +import redis |
| 52 | + |
| 53 | +r = redis.Redis(decode_responses=True) |
| 54 | + |
| 55 | +pipe = r.pipeline() |
| 56 | + |
| 57 | +for i in range(5): |
| 58 | + pipe.set(f"seat:{i}", f"#{i}") |
| 59 | + |
| 60 | +set_5_result = pipe.execute() |
| 61 | +print(set_5_result) # >>> [True, True, True, True, True] |
| 62 | + |
| 63 | +pipe = r.pipeline() |
| 64 | + |
| 65 | +# "Chain" pipeline commands together. |
| 66 | +get_3_result = pipe.get("seat:0").get("seat:3").get("seat:4").execute() |
| 67 | +print(get_3_result) # >>> ['#0', '#3', '#4'] |
| 68 | +``` |
| 69 | + |
| 70 | +## Execute a transaction |
| 71 | + |
| 72 | +A pipeline actually executes as a transaction by default (that is to say, |
| 73 | +all commands are executed in an uninterrupted sequence). However, if you |
| 74 | +need to switch this behavior off, you can set the `transaction` parameter |
| 75 | +to `False` when you create the pipeline: |
| 76 | + |
| 77 | +```python |
| 78 | +pipe = r.pipeline(transaction=False) |
| 79 | +``` |
| 80 | + |
| 81 | +## Watch keys for changes |
| 82 | + |
| 83 | +When you use transactions, you will often need to read values from the |
| 84 | +database, process them, and then write the modified values back. Ideally, |
| 85 | +you would want to perform the whole read-modify-write sequence atomically, without any |
| 86 | +interruptions from other clients. However, you don't get the results from |
| 87 | +any commands that are buffered in a transaction until the whole transaction has finished |
| 88 | +executing. This means you can't read keys, process the data, and then write the keys back |
| 89 | +in the same transaction. Other clients can therefore modify the values you have |
| 90 | +read before you have the chance to write them. |
| 91 | + |
| 92 | +Redis solves this problem by letting you watch for changes to keys in the |
| 93 | +database just before executing a transaction. The basic stages are as |
| 94 | +follows: |
| 95 | + |
| 96 | +1. Start watching the keys you are about to update. |
| 97 | +1. Read the data values from those keys. |
| 98 | +1. Perform changes to the data values. |
| 99 | +1. Add commands to a transaction to write the data values back. |
| 100 | +1. Attempt to execute the transaction. |
| 101 | +1. If the keys you were watching changed before the transaction started |
| 102 | + executing, then abort the transaction and start again from step 1. |
| 103 | + Otherwise, the transaction was successful and the updated data is written back. |
| 104 | + |
| 105 | +This technique is called *optimistic locking* and works well in cases |
| 106 | +where multiple clients might access the same data simultaneously, but |
| 107 | +usually don't. See |
| 108 | +[Transactions]({{< relref "/develop/interact/transactions" >}}) |
| 109 | +for more information about optimistic locking. |
| 110 | + |
| 111 | +The example below shows how to repeatedly attempt a transaction with a watched |
| 112 | +key until it succeeds. The code reads a string |
| 113 | +that represents a `PATH` variable for a command shell, then appends a new |
| 114 | +command path to the string before attempting to write it back. If the watched |
| 115 | +key is modified by another client before writing, the transaction aborts |
| 116 | +with a `WatchError` exception, and the loop executes again for another attempt. |
| 117 | +Otherwise, the loop terminates successfully. |
| 118 | + |
| 119 | +<!-- |
| 120 | +{{< clients-example pipe_trans_tutorial trans_watch Python >}} |
| 121 | +{{< /clients-example >}} |
| 122 | +--> |
| 123 | +```python |
| 124 | +r.set("shellpath", "/usr/syscmds/") |
| 125 | + |
| 126 | +with r.pipeline() as pipe: |
| 127 | + # Repeat until successful. |
| 128 | + while True: |
| 129 | + try: |
| 130 | + # Watch the key we are about to change. |
| 131 | + pipe.watch("shellpath") |
| 132 | + |
| 133 | + # The pipeline executes commands directly (instead of |
| 134 | + # buffering them) from immediately after the `watch()` |
| 135 | + # call until we begin the transaction. |
| 136 | + current_path = pipe.get("shellpath") |
| 137 | + new_path = current_path + ":/usr/mycmds/" |
| 138 | + |
| 139 | + # Start the transaction, which will enable buffering |
| 140 | + # again for the remaining commands. |
| 141 | + pipe.multi() |
| 142 | + |
| 143 | + pipe.set("shellpath", new_path) |
| 144 | + |
| 145 | + pipe.execute() |
| 146 | + |
| 147 | + # The transaction succeeded, so break out of the loop. |
| 148 | + break |
| 149 | + except redis.WatchError: |
| 150 | + # The transaction failed, so continue with the next attempt. |
| 151 | + continue |
| 152 | + |
| 153 | +get_path_result = r.get("shellpath") |
| 154 | +print(get_path_result) # >>> '/usr/syscmds/:/usr/mycmds/' |
| 155 | +``` |
| 156 | + |
| 157 | +Because this is a common pattern, the library includes a convenience |
| 158 | +method called `transaction()` that handles the code to watch keys, |
| 159 | +execute the transaction, and retry if necessary. Pass |
| 160 | +`transaction()` a function that implements your main transaction code, |
| 161 | +and also pass the keys you want to watch. The example below implements |
| 162 | +the same basic transaction as the previous example but this time |
| 163 | +using `transaction()`. Note that `transaction()` can't add the `multi()` |
| 164 | +call automatically, so you must still place this correctly in your |
| 165 | +transaction function. |
| 166 | + |
| 167 | +<!-- |
| 168 | +{{< clients-example pipe_trans_tutorial watch_conv_method Python >}} |
| 169 | +{{< /clients-example >}} |
| 170 | +*--> |
| 171 | +```python |
| 172 | +r.set("shellpath", "/usr/syscmds/") |
| 173 | + |
| 174 | + |
| 175 | +def watched_sequence(pipe): |
| 176 | + current_path = pipe.get("shellpath") |
| 177 | + new_path = current_path + ":/usr/mycmds/" |
| 178 | + |
| 179 | + pipe.multi() |
| 180 | + |
| 181 | + pipe.set("shellpath", new_path) |
| 182 | + |
| 183 | + |
| 184 | +trans_result = r.transaction(watched_sequence, "shellpath") |
| 185 | +print(trans_result) # True |
| 186 | + |
| 187 | +get_path_result = r.get("shellpath") |
| 188 | +print(get_path_result) # >>> '/usr/syscmds/:/usr/mycmds/' |
| 189 | +``` |
0 commit comments