-
Notifications
You must be signed in to change notification settings - Fork 713
/
Copy path7_Serialization.cs
471 lines (399 loc) · 22.5 KB
/
7_Serialization.cs
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT license.
using System;
using System.Collections.Generic;
using System.IO;
using Microsoft.Research.SEAL;
namespace SEALNetExamples
{
partial class Examples
{
/*
In this example we show how serialization works in Microsoft SEAL. Specifically,
we present important concepts that enable the user to optimize the data size when
communicating ciphertexts and keys for outsourced computation. Unlike the previous
examples, we organize this one in a client-server style for maximal clarity. The
server selects encryption parameters, the client generates keys, the server does
the encrypted computation, and the client decrypts.
*/
private static void ExampleSerialization()
{
Utilities.PrintExampleBanner("Example: Serialization");
/*
We require ZLIB or Zstandard support for this example to be available.
*/
if (!Serialization.IsSupportedComprMode(ComprModeType.ZLIB) &&
!Serialization.IsSupportedComprMode(ComprModeType.ZSTD))
{
Console.WriteLine("Neither ZLIB nor Zstandard support is enabled; this example is not available.");
Console.WriteLine();
return;
}
/*
We start by briefly discussing the Serializable<T> generic class. This is
a wrapper class that can wrap any serializable class, which include:
- EncryptionParameters
- Modulus
- Plaintext and Ciphertext
- SecretKey, PublicKey, RelinKeys, and GaloisKeys
Serializable<T> provides minimal functionality needed to serialize the wrapped
object by simply forwarding the calls to corresponding functions of the wrapped
object of type T. The need for Serializable<T> comes from the fact that many
Microsoft SEAL objects consist of two parts, one of which is pseudorandom data
independent of the other part. Until the object is actually being used, the
pseudorandom part can be instead stored as a seed. We will call objects with
property `seedable'.
For example, GaloisKeys can often be very large in size, but in reality half
of the data is pseudorandom and can be stored as a seed. Since GaloisKeys are
never used by the party that generates them, so it makes sense to expand the
seed at the point deserialization. On the other hand, we cannot allow the user
to accidentally try to use an unexpanded GaloisKeys object, which is prevented
at by ensuring it is always wrapped in a Serializable<GaloisKeys> and can only
be serialized.
Only some Microsoft SEAL objects are seedable. Specifically, they are:
- PublicKey, RelinKeys, and GaloisKeys
- Ciphertext in secret-key mode (from Encryptor.EncryptSymmetric or
Encryptor.EncryptZeroSymmetric)
Importantly, ciphertexts in public-key mode are not seedable. Thus, it may
be beneficial to use Microsoft SEAL in secret-key mode whenever the public
key is not truly needed.
There are a handful of functions that output Serializable<T> objects:
- Encryptor.Encrypt (and variants) output Serializable<Ciphertext>
- KeyGenerator.Create... output Serializable<T> for different key types
Note that Encryptor.Encrypt is included in the above list, yet it produces
ciphertexts in public-key mode that are not seedable. This is for the sake of
consistency in the API for public-key and secret-key encryption. Functions
that output Serializable<T> objects also have overloads that take a normal
object of type T as a destination parameter, overwriting it. These overloads
can be convenient for local testing where no serialization is needed and the
object needs to be used at the point of construction. Such an object can no
longer be transformed back to a seeded state.
*/
/*
To simulate client-server interaction, we set up a shared C# stream. In real
use-cases this can be a network stream, a filestream, or any shared resource.
It is critical to note that all data serialized by Microsoft SEAL is in binary
form, so it is not meaningful to print the data as ASCII characters. Encodings
such as Base64 would increase the data size, which is already a bottleneck in
homomorphic encryption. Hence, serialization into text is not supported or
recommended.
In this example we use a couple of shared MemoryStreams.
*/
using MemoryStream parmsStream = new MemoryStream();
using MemoryStream dataStream = new MemoryStream();
using MemoryStream skStream = new MemoryStream();
/*
The server first determines the computation and sets encryption parameters
accordingly.
*/
{
ulong polyModulusDegree = 8192;
using EncryptionParameters parms = new EncryptionParameters(SchemeType.CKKS);
parms.PolyModulusDegree = polyModulusDegree;
parms.CoeffModulus = CoeffModulus.Create(
polyModulusDegree, new int[]{ 50, 30, 50 });
/*
Serialization of the encryption parameters to our shared stream is very
simple with the EncryptionParameters.Save function.
*/
long size = parms.Save(parmsStream);
/*
Seek the parmsStream head back to beginning of the stream.
*/
parmsStream.Seek(0, SeekOrigin.Begin);
/*
The return value of this function is the actual byte count of data written
to the stream.
*/
Utilities.PrintLine();
Console.WriteLine($"EncryptionParameters: wrote {size} bytes");
/*
Before moving on, we will take some time to discuss further options in
serialization. These will become particularly important when the user
needs to optimize communication and storage sizes.
It is possible to enable or disable compression for serialization by
providing EncryptionParameters.Save with the desired compression mode as
in the following examples:
long size = parms.Save(sharedStream, ComprModeType.None);
long size = parms.Save(sharedStream, ComprModeType.ZLIB);
long size = parms.Save(sharedStream, ComprModeType.ZSTD);
If Microsoft SEAL is compiled with Zstandard or ZLIB support, the default
is to use one of them. If available, Zstandard is preferred over ZLIB due
to its speed.
Compression can have a substantial impact on the serialized data size,
because ciphertext and key data consists of many uniformly random integers
modulo the CoeffModulus primes. Especially when using CKKS, the primes in
CoeffModulus can be relatively small compared to the 64-bit words used to
store the ciphertext and key data internally. Serialization writes full
64-bit words to the destination buffer or stream, possibly leaving in many
zero bytes corresponding to the high-order bytes of the 64-bit words. One
convenient way to get rid of these zeros is to apply a general-purpose
compression algorithm on the encrypted data. The compression rate can be
significant (up to 50-60%) when using CKKS with small primes.
*/
/*
In many cases, when working with fixed size memory, it is necessary to know
ahead of time an upper bound on the serialized data size to allocate enough
memory. This information is returned by the EncryptionParameters.SaveSize
function. This function accepts the desired compression mode, or uses the
default option otherwise.
In more detail, the output of EncryptionParameters.SaveSize is as follows:
- Exact buffer size required for ComprModeType.None;
- Upper bound on the size required for ComprModeType.ZLIB or
ComprModeType.ZSTD.
As we can see from the print-out, the sizes returned by these functions
are significantly larger than the compressed size written into the shared
stream in the beginning. This is normal: compression yielded a significant
improvement in the data size, however, it is impossible to know ahead of
time the exact size of the compressed data. If compression is not used,
then the size is exactly determined by the encryption parameters.
*/
Utilities.PrintLine();
Console.Write("EncryptionParameters: data size upper bound (ComprModeType.None): ");
Console.WriteLine(parms.SaveSize(ComprModeType.None));
Console.Write(" ");
Console.Write("EncryptionParameters: data size upper bound (compression): ");
Console.WriteLine(parms.SaveSize(/* Serialization.ComprModeDefault */));
/*
As an example, we now serialize the encryption parameters to a fixed
size buffer.
*/
using MemoryStream buffer = new MemoryStream(new byte[parms.SaveSize()]);
parms.Save(buffer);
/*
To illustrate deserialization, we load back the encryption parameters
from our buffer into another instance of EncryptionParameters. First
we need to seek our stream back to the beginning.
*/
buffer.Seek(0, SeekOrigin.Begin);
using EncryptionParameters parms2 = new EncryptionParameters();
parms2.Load(buffer);
/*
We can check that the saved and loaded encryption parameters indeed match.
*/
Utilities.PrintLine();
Console.WriteLine($"EncryptionParameters: parms == parms2: {parms.Equals(parms2)}");
}
/*
Client starts by loading the encryption parameters, sets up the SEALContext,
and creates the required keys.
*/
{
using EncryptionParameters parms = new EncryptionParameters();
parms.Load(parmsStream);
/*
Seek the parmsStream head back to beginning of the stream because we
will use the same stream to read the parameters repeatedly.
*/
parmsStream.Seek(0, SeekOrigin.Begin);
using SEALContext context = new SEALContext(parms);
using KeyGenerator keygen = new KeyGenerator(context);
using SecretKey sk = keygen.SecretKey;
keygen.CreatePublicKey(out PublicKey pk);
/*
We need to save the secret key so we can decrypt later.
*/
sk.Save(skStream);
skStream.Seek(0, SeekOrigin.Begin);
/*
As in previous examples, in this example we will encrypt in public-key
mode. If we want to send a public key over the network, we should instead
have created it as a seeded object as follows:
Serializable<PublicKey> pk = keygen.CreatePublicKey();
In this example we will also use relinearization keys. These we will
absolutely want to create as seeded objects to minimize communication
cost, unlike in prior examples.
*/
using Serializable<RelinKeys> rlk = keygen.CreateRelinKeys();
/*
To demonstrate the significant space saving from this method, we will
create another set of relinearization keys, this time fully expanded.
*/
keygen.CreateRelinKeys(out RelinKeys rlkBig);
/*
We serialize both relinearization keys to demonstrate the concrete size
difference. If compressed serialization is used, the compression rate
will be the same in both cases. We omit specifying the compression mode
to use the default, as determined by the Microsoft SEAL build system.
*/
long sizeRlk = rlk.Save(dataStream);
long sizeRlkBig = rlkBig.Save(dataStream);
Utilities.PrintLine();
Console.WriteLine($"Serializable<RelinKeys>: wrote {sizeRlk} bytes");
Console.Write(" ");
Console.WriteLine($"RelinKeys: wrote {sizeRlkBig} bytes");
/*
Seek back in dataStream to where rlk data ended, i.e., sizeRlkBig bytes
backwards from current position.
*/
dataStream.Seek(-sizeRlkBig, SeekOrigin.Current);
/*
Next set up the CKKSEncoder and Encryptor, and encrypt some numbers.
*/
double scale = Math.Pow(2.0, 30);
CKKSEncoder encoder = new CKKSEncoder(context);
using Plaintext plain1 = new Plaintext(),
plain2 = new Plaintext();
encoder.Encode(2.3, scale, plain1);
encoder.Encode(4.5, scale, plain2);
using Encryptor encryptor = new Encryptor(context, pk);
/*
The client will not compute on ciphertexts that it creates, so it can
just as well create Serializable<Ciphertext> objects. In fact, we do
not even need to name those objects and instead immediately call
Serializable<Ciphertext>.Save.
*/
long sizeEncrypted1 = encryptor.Encrypt(plain1).Save(dataStream);
/*
As we discussed in the beginning of this example, ciphertexts can
be created in a seeded state in secret-key mode, providing a huge
reduction in the data size upon serialization. To do this, we need
to provide the Encryptor with the secret key in its constructor, or
at a later point with the Encryptor.SetSecretKey function, and use
the Encryptor.EncryptSymmetric function to encrypt.
*/
encryptor.SetSecretKey(sk);
long sizeSymEncrypted2 = encryptor.EncryptSymmetric(plain2).Save(dataStream);
/*
The size reduction is substantial.
*/
Utilities.PrintLine();
Console.WriteLine($"Serializable<Ciphertext> (public-key): wrote {sizeEncrypted1} bytes");
Console.Write(" ");
Console.Write($"Serializable<Ciphertext> (seeded secret-key): ");
Console.WriteLine($"wrote {sizeSymEncrypted2} bytes");
/*
Seek to the beginning of dataStream.
*/
dataStream.Seek(0, SeekOrigin.Begin);
/*
We have seen how creating seeded objects can result in huge space
savings compared to creating unseeded objects. This is particularly
important when creating Galois keys, which can be very large. We have
seen how secret-key encryption can be used to achieve much smaller
ciphertext sizes when the public-key functionality is not needed.
We would also like to draw attention to the fact there we could easily
serialize multiple Microsoft SEAL objects sequentially in a stream. Each
object writes its own size into the stream, so deserialization knows
exactly how many bytes to read. We will see this working below.
*/
}
/*
The server can now compute on the encrypted data. We will recreate the
SEALContext and set up an Evaluator here.
*/
{
using EncryptionParameters parms = new EncryptionParameters();
parms.Load(parmsStream);
parmsStream.Seek(0, SeekOrigin.Begin);
using SEALContext context = new SEALContext(parms);
using Evaluator evaluator = new Evaluator(context);
/*
Next we need to load relinearization keys and the ciphertexts from our
dataStream.
*/
using RelinKeys rlk = new RelinKeys();
using Ciphertext encrypted1 = new Ciphertext(),
encrypted2 = new Ciphertext();
/*
Deserialization is as easy as serialization.
*/
rlk.Load(context, dataStream);
encrypted1.Load(context, dataStream);
encrypted2.Load(context, dataStream);
/*
Compute the product, rescale, and relinearize.
*/
using Ciphertext encryptedProd = new Ciphertext();
evaluator.Multiply(encrypted1, encrypted2, encryptedProd);
evaluator.RelinearizeInplace(encryptedProd, rlk);
evaluator.RescaleToNextInplace(encryptedProd);
/*
We use dataStream to communicate encryptedProd back to the client.
There is no way to save the encryptedProd as a seeded object: only
freshly encrypted secret-key ciphertexts can be seeded. Note how the
size of the result ciphertext is smaller than the size of a fresh
ciphertext because it is at a lower level due to the rescale operation.
*/
dataStream.Seek(0, SeekOrigin.Begin);
long sizeEncryptedProd = encryptedProd.Save(dataStream);
dataStream.Seek(0, SeekOrigin.Begin);
Utilities.PrintLine();
Console.Write($"Ciphertext (secret-key): ");
Console.WriteLine($"wrote {sizeEncryptedProd} bytes");
}
/*
In the final step the client decrypts the result.
*/
{
using EncryptionParameters parms = new EncryptionParameters();
parms.Load(parmsStream);
parmsStream.Seek(0, SeekOrigin.Begin);
using SEALContext context = new SEALContext(parms);
/*
Load back the secret key from skStream.
*/
using SecretKey sk = new SecretKey();
sk.Load(context, skStream);
using Decryptor decryptor = new Decryptor(context, sk);
using CKKSEncoder encoder = new CKKSEncoder(context);
using Ciphertext encryptedResult = new Ciphertext();
encryptedResult.Load(context, dataStream);
using Plaintext plainResult = new Plaintext();
decryptor.Decrypt(encryptedResult, plainResult);
List<double> result = new List<double>();
encoder.Decode(plainResult, result);
Utilities.PrintLine();
Console.WriteLine("Decrypt and decode PI * x ^ 3 + 0.4x + 1.");
Console.WriteLine(" + Expected result:");
List<double> trueResult = new List<double>((int)encoder.SlotCount);
for (ulong i = 0; i < encoder.SlotCount; i++)
{
trueResult.Add(2.3 * 4.5);
}
Utilities.PrintVector(trueResult, 3, 7);
Console.WriteLine(" + Computed result ...... Correct.");
Utilities.PrintVector(result, 3, 7);
}
/*
Finally, we give a little bit more explanation of the structure of data
serialized by Microsoft SEAL. Serialized data always starts with a 16-byte
SEALHeader struct, as defined in dotnet/src/Serialization.cs, and is
followed by the possibly compressed data for the object.
A SEALHeader contains the following data:
[offset 0] 2-byte magic number 0xA15E (Serialization.SEALMagic)
[offset 2] 1-byte indicating the header size in bytes (always 16)
[offset 3] 1-byte indicating the Microsoft SEAL major version number
[offset 4] 1-byte indicating the Microsoft SEAL minor version number
[offset 5] 1-byte indicating the compression mode type
[offset 6] 2-byte reserved field (unused)
[offset 8] 8-byte size in bytes of the serialized data, including the header
Currently Microsoft SEAL supports only little-endian systems.
As an example, we demonstrate the SEALHeader created by saving a plaintext.
Note that the SEALHeader is never compressed, so there is no need to specify
the compression mode.
*/
using Plaintext pt = new Plaintext("1x^2 + 3");
using MemoryStream stream = new MemoryStream();
long dataSize = pt.Save(stream);
/*
Seek the stream head back to beginning of the stream.
*/
stream.Seek(0, SeekOrigin.Begin);
/*
We can now load just the SEALHeader back from the stream as follows.
*/
Serialization.SEALHeader header = new Serialization.SEALHeader();
Serialization.LoadHeader(stream, header);
/*
Now confirm that the size of data written to stream matches with what is
indicated by the SEALHeader.
*/
Utilities.PrintLine();
Console.WriteLine($"Size written to stream: {dataSize} bytes");
Console.Write(" ");
Console.WriteLine($"Size indicated in SEALHeader: {header.Size} bytes");
Console.WriteLine();
}
}
}