Serialization#
Rateslib is an object oriented library requiring dynamic associations to create pricing and risk functionality. The objective of serialization and deserialization is to convert rateslib’s objects into a form that can be persisted or transported, with a view towards being able to recapture those dynamic associations (i.e. saved and loaded from a database, or sent between client and server over a network).
Warning
This feature is currently experimental, and in development. This page documents the objects that have, to date, had focused development for the two forms of serialization rateslib intends to offer. Other objects in rateslib may well have serialization methods in place but the stability and robustness of those may not have been properly tested.
Forms#
Serialization is a general term. The specific forms of serialization that rateslib aims to create is JSON serialization and Python pickling.
JSON serialization converts an object to a string and reconstructs it from that same string. The advantage of JSON is that it is human readable (and often editable) whilst a disadvantage is that it likely to be a lesser efficient form of transfer or storage. It is considered a safe form of serialization.
In [1]: from rateslib import from_json
Create an object from JSON string. |
Pickling is a native conversion of binary objects to byte arrays. That same array can be reconstituted into a Python object. This is typically very efficient but the resultant arrays of data are not human-readable (or editable). This is considered an unsafe form of serialization due to the arbitrary nature of code execution upon deserialization.
Pickling of data is often performed internally by Python’s multiprocessing module if it needs to share data across threads.
In [2]: from pickle import dumps, loads
Objects in Scope#
We aim to keep the objects in scope synchronized with availability for both json and pickle serialization for the stated objects below.
Serializing
# --------------dual ---------------------
In [3]: dual.to_json() # Dual
Out[3]: '{"Dual":{"real":3.141,"vars":["x","y"],"dual":{"v":1,"dim":[2],"data":[1.0,0.0]}}}'
In [4]: dual2.to_json() # Dual2
Out[4]: '{"Dual2":{"real":3.141,"vars":["x","y"],"dual":{"v":1,"dim":[2],"data":[1.0,0.0]},"dual2":{"v":1,"dim":[2,2],"data":[0.0,0.0,0.0,0.0]}}}'
In [5]: variable.to_json() # Variable
Out[5]: '{"PyNative": {"Variable": {"real": 3.141, "vars": ["z"], "dual": [1.0]}}}'
# ---------- scheduling ------------------
In [6]: cal.to_json() # Cal
Out[6]: '{"Cal":{"holidays":[],"week_mask":["Sat","Sun"]}}'
In [7]: union_cal.to_json() # UnionCal
Out[7]: '{"UnionCal":{"calendars":[{"holidays":[],"week_mask":["Sat","Sun"]}],"settlement_calendars":[]}}'
In [8]: named_cal.to_json() # NamedCal
Out[8]: '{"NamedCal":{"name":"bus"}}'
In [9]: adjuster.to_json() # Adjuster
Out[9]: '{"PyAdjuster":{"Following":{"_u8":1}}}'
In [10]: imm.to_json() # Imm
Out[10]: '{"Imm":"Wed3"}'
In [11]: roll.to_json() # RollDay
Out[11]: '{"RollDay":{"Day":31}}'
In [12]: frequency.to_json() # Frequency
Out[12]: '{"Frequency":{"Months":{"number":4,"roll":{"Day":31}}}}'
In [13]: stub_inference.to_json() # StubInference
Out[13]: '{"StubInference":"LongFront"}'
In [14]: schedule.to_json() # Schedule
Out[14]: '{"PyWrapped":{"Schedule":{"ueffective":"2000-01-01T00:00:00","utermination":"2001-01-01T00:00:00","frequency":{"Months":{"number":6,"roll":{"Day":1}}},"ufront_stub":null,"uback_stub":null,"calendar":{"NamedCal":{"name":"all"}},"accrual_adjuster":{"ModifiedFollowing":{}},"payment_adjuster":{"BusDaysLagSettle":2}}}}'
# ------------ splines --------------------
In [15]: ppspline_f64.to_json() # PPSplineF64
Out[15]: '{"PPSplineF64":{"inner":{"k":4,"t":[0.0,0.0,0.0,0.0,4.0,4.0,4.0,4.0],"c":{"v":1,"dim":[4],"data":[0.3,0.2,0.6,0.2]},"n":4}}}'
In [16]: ppspline_dual.to_json() # PPSplineDual
Out[16]: '{"PPSplineDual":{"inner":{"k":3,"t":[0.0,0.0,0.0,1.0,1.0,1.0],"c":{"v":1,"dim":[3],"data":[{"real":0.1,"vars":[],"dual":{"v":1,"dim":[0],"data":[]}},{"real":0.2,"vars":[],"dual":{"v":1,"dim":[0],"data":[]}},{"real":0.3,"vars":[],"dual":{"v":1,"dim":[0],"data":[]}}]},"n":3}}}'
In [17]: ppspline_dual2.to_json() # PPSplineDual2
Out[17]: '{"PPSplineDual2":{"inner":{"k":3,"t":[0.0,0.0,0.0,1.0,1.0,1.0],"c":{"v":1,"dim":[3],"data":[{"real":0.1,"vars":[],"dual":{"v":1,"dim":[0],"data":[]},"dual2":{"v":1,"dim":[0,0],"data":[]}},{"real":0.2,"vars":[],"dual":{"v":1,"dim":[0],"data":[]},"dual2":{"v":1,"dim":[0,0],"data":[]}},{"real":0.3,"vars":[],"dual":{"v":1,"dim":[0],"data":[]},"dual2":{"v":1,"dim":[0,0],"data":[]}}]},"n":3}}}'
# ------------ fx_rates -------------------
In [18]: fxr.to_json() # FXRates
Out[18]: '{"PyWrapped":{"FXRates":{"fx_rates":[{"pair":[{"name":"eur"},{"name":"usd"}],"rate":{"F64":1.15},"settlement":"2000-01-01T00:00:00"}],"currencies":[{"name":"eur"},{"name":"usd"}]}}}'
Deserializing
# --------------dual ---------------------
In [19]: from_json(dual_json)
Out[19]: <Dual: 3.141000, (x, y), [1.0, 0.0]>
In [20]: from_json(dual2_json)
Out[20]: <Dual2: 3.141000, (x, y), [1.0, 0.0], [[...]]>
In [21]: from_json(variable_json)
Out[21]: <Variable: 3.141, (z), [1.0]>
# ---------- scheduling ------------------
In [22]: from_json(cal_json)
Out[22]: <rateslib.rs.Cal at 0x12210b2d0>
In [23]: from_json(union_cal_json)
Out[23]: <rateslib.rs.UnionCal at 0x1221a7b90>
In [24]: from_json(named_cal_json)
Out[24]: <rateslib.rs.NamedCal at 0x12219e4f0>
In [25]: from_json(adjuster_json)
Out[25]: <rl.Adjuster.Following at 0x122178fc0>
In [26]: from_json(imm_json)
Out[26]: <rl.Imm.Wed3 at 0x122179000>
In [27]: from_json(roll_json)
Out[27]: <rl.RollDay.Day(31) at 0x122179060>
In [28]: from_json(frequency_json)
Out[28]: <rl.Frequency.Months(4, Day(31)) at 0x12210aa70>
In [29]: from_json(stub_inference_json)
Out[29]: <rl.StubInference.LongFront at 0x122179120>
In [30]: from_json(schedule_json)
Out[30]: <rl.Schedule at 0x121db5860>
# ------------ splines --------------------
In [31]: from_json(ppspline_f64_json)
Out[31]: <rl.PPSplineF64 at 0x12218e730>
In [32]: from_json(ppspline_dual_json)
Out[32]: <rl.PPSplineDual at 0x12218e5e0>
In [33]: from_json(ppspline_dual2_json)
Out[33]: <rl.PPSplineDual2 at 0x121da0f20>
# ------------ fx_rates -------------------
In [34]: from_json(fxr_json)
Out[34]: <rl.FXRates:[eur,usd] at 0x121da09f0>
Serializing
# --------------dual ---------------------
In [35]: dumps(dual) # Dual
Out[35]: b'\x80\x04\x95G\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\x04Dual\x94\x93\x94G@\t \xc4\x9b\xa5\xe3T]\x94(\x8c\x01x\x94\x8c\x01y\x94e]\x94(G?\xf0\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00e\x87\x94\x81\x94.'
In [36]: dumps(dual2) # Dual2
Out[36]: b'\x80\x04\x95q\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\x05Dual2\x94\x93\x94(G@\t \xc4\x9b\xa5\xe3T]\x94(\x8c\x01x\x94\x8c\x01y\x94e]\x94(G?\xf0\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00e]\x94(G\x00\x00\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00et\x94\x81\x94.'
In [37]: dumps(variable) # Variable
Out[37]: b'\x80\x04\x95\xe3\x00\x00\x00\x00\x00\x00\x00\x8c\x16rateslib.dual.variable\x94\x8c\x08Variable\x94\x93\x94)\x81\x94}\x94(\x8c\x04real\x94G@\t \xc4\x9b\xa5\xe3T\x8c\x04vars\x94\x8c\x01z\x94\x85\x94\x8c\x04dual\x94\x8c\x16numpy._core.multiarray\x94\x8c\x0c_reconstruct\x94\x93\x94\x8c\x05numpy\x94\x8c\x07ndarray\x94\x93\x94K\x00\x85\x94C\x01b\x94\x87\x94R\x94(K\x01K\x01\x85\x94h\r\x8c\x05dtype\x94\x93\x94\x8c\x02f8\x94\x89\x88\x87\x94R\x94(K\x03\x8c\x01<\x94NNNJ\xff\xff\xff\xffJ\xff\xff\xff\xffK\x00t\x94b\x89C\x08\x00\x00\x00\x00\x00\x00\xf0?\x94t\x94bub.'
# ---------- scheduling ------------------
In [38]: dumps(cal) # Cal
Out[38]: b'\x80\x04\x95"\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\x03Cal\x94\x93\x94]\x94C\x02\x05\x06\x94\x86\x94\x81\x94.'
In [39]: dumps(union_cal) # UnionCal
Out[39]: b'\x80\x04\x95F\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\x08UnionCal\x94\x93\x94]\x94\x8c\x0brateslib.rs\x94\x8c\x03Cal\x94\x93\x94]\x94C\x02\x05\x06\x94\x86\x94\x81\x94a]\x94\x86\x94\x81\x94.'
In [40]: dumps(named_cal) # NamedCal
Out[40]: b'\x80\x04\x95&\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\x08NamedCal\x94\x93\x94\x8c\x03bus\x94\x85\x94\x81\x94.'
In [41]: dumps(adjuster) # Adjuster
Out[41]: b'\x80\x04\x95.\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\x14PyAdjuster_Following\x94\x93\x94K\x01\x85\x94\x81\x94.'
In [42]: dumps(imm) # Imm
Out[42]: b'\x80\x04\x95\x1d\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\x03Imm\x94\x93\x94K\x01\x85\x94\x81\x94.'
In [43]: dumps(roll) # RollDay
Out[43]: b'\x80\x04\x95%\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\x0bRollDay_Day\x94\x93\x94K\x1f\x85\x94\x81\x94.'
In [44]: dumps(frequency) # Frequency
Out[44]: b'\x80\x04\x95N\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\x10Frequency_Months\x94\x93\x94K\x04\x8c\x0brateslib.rs\x94\x8c\x0bRollDay_Day\x94\x93\x94K\x1f\x85\x94\x81\x94\x86\x94\x81\x94.'
In [45]: dumps(stub_inference) # StubInference
Out[45]: b"\x80\x04\x95'\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\rStubInference\x94\x93\x94K\x01\x85\x94\x81\x94."
In [46]: dumps(schedule) # Schedule
Out[46]: b'\x80\x04\x95\x86\x02\x00\x00\x00\x00\x00\x00\x8c\x1crateslib.scheduling.schedule\x94\x8c\x08Schedule\x94\x93\x94(\x8c\x08datetime\x94\x8c\x08datetime\x94\x93\x94C\n\x07\xd0\x01\x01\x00\x00\x00\x00\x00\x00\x94\x85\x94R\x94h\x05C\n\x07\xd1\x01\x01\x00\x00\x00\x00\x00\x00\x94\x85\x94R\x94\x8c\x0brateslib.rs\x94\x8c\x10Frequency_Months\x94\x93\x94K\x06\x8c\x0brateslib.rs\x94\x8c\x0bRollDay_Day\x94\x93\x94K\x01\x85\x94\x81\x94\x86\x94\x81\x94\x8c\x10rateslib.default\x94\x8c\x07NoInput\x94\x93\x94K\x00\x85\x94R\x94h\x1ah\x1ah\x1ah\x1a\x8c\x0brateslib.rs\x94\x8c\x1cPyAdjuster_ModifiedFollowing\x94\x93\x94K\x02\x85\x94\x81\x94\x8c\x0brateslib.rs\x94\x8c\x08NamedCal\x94\x93\x94\x8c\x03all\x94\x85\x94\x81\x94\x8c\x0brateslib.rs\x94\x8c\x1bPyAdjuster_BusDaysLagSettle\x94\x93\x94K\x02K\t\x86\x94\x81\x94h\x1ah\x1at\x94\x81\x94}\x94(\x8c\x04_obj\x94\x8c\x0brateslib.rs\x94\x8c\x08Schedule\x94\x93\x94(h\x05C\n\x07\xd0\x01\x01\x00\x00\x00\x00\x00\x00\x94\x85\x94R\x94h\x05C\n\x07\xd1\x01\x01\x00\x00\x00\x00\x00\x00\x94\x85\x94R\x94h\x0eK\x06h\x11K\x01\x85\x94\x81\x94\x86\x94\x81\x94h"\x8c\x03all\x94\x85\x94\x81\x94h\x1dK\x02\x85\x94\x81\x94h(K\x02K\t\x86\x94\x81\x94\x89NNNt\x94\x81\x94\x8c\nueffective\x94h\x08\x8c\x0cutermination\x94h\x0b\x8c\rfrequency_obj\x94h\x15\x8c\x0bufront_stub\x94N\x8c\nuback_stub\x94N\x8c\x10accrual_adjuster\x94h\x1f\x8c\x08calendar\x94h%\x8c\x10payment_adjuster\x94h*ub.'
# ------------ splines --------------------
In [47]: dumps(ppspline_f64) # PPSplineF64
Out[47]: b'\x80\x04\x95\x99\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\x0bPPSplineF64\x94\x93\x94K\x04]\x94(G\x00\x00\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00G@\x10\x00\x00\x00\x00\x00\x00G@\x10\x00\x00\x00\x00\x00\x00G@\x10\x00\x00\x00\x00\x00\x00G@\x10\x00\x00\x00\x00\x00\x00e]\x94(G?\xd3333333G?\xc9\x99\x99\x99\x99\x99\x9aG?\xe3333333G?\xc9\x99\x99\x99\x99\x99\x9ae\x87\x94\x81\x94.'
In [48]: dumps(ppspline_dual) # PPSplineDual
Out[48]: b'\x80\x04\x95\xb2\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\x0cPPSplineDual\x94\x93\x94K\x03]\x94(G\x00\x00\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00G?\xf0\x00\x00\x00\x00\x00\x00G?\xf0\x00\x00\x00\x00\x00\x00G?\xf0\x00\x00\x00\x00\x00\x00e]\x94(\x8c\x0brateslib.rs\x94\x8c\x04Dual\x94\x93\x94G?\xb9\x99\x99\x99\x99\x99\x9a]\x94]\x94\x87\x94\x81\x94h\x07G?\xc9\x99\x99\x99\x99\x99\x9a]\x94]\x94\x87\x94\x81\x94h\x07G?\xd3333333]\x94]\x94\x87\x94\x81\x94e\x87\x94\x81\x94.'
In [49]: dumps(ppspline_dual2) # PPSplineDual2
Out[49]: b'\x80\x04\x95\xbd\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\rPPSplineDual2\x94\x93\x94K\x03]\x94(G\x00\x00\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00G?\xf0\x00\x00\x00\x00\x00\x00G?\xf0\x00\x00\x00\x00\x00\x00G?\xf0\x00\x00\x00\x00\x00\x00e]\x94(\x8c\x0brateslib.rs\x94\x8c\x05Dual2\x94\x93\x94(G?\xb9\x99\x99\x99\x99\x99\x9a]\x94]\x94]\x94t\x94\x81\x94h\x07(G?\xc9\x99\x99\x99\x99\x99\x9a]\x94]\x94]\x94t\x94\x81\x94h\x07(G?\xd3333333]\x94]\x94]\x94t\x94\x81\x94e\x87\x94\x81\x94.'
# ------------ fx_rates -------------------
In [50]: dumps(fxr) # FXRates
Out[50]: b'\x80\x04\x95\xbd\x01\x00\x00\x00\x00\x00\x00\x8c\x14rateslib.fx.fx_rates\x94\x8c\x07FXRates\x94\x93\x94)\x81\x94}\x94(\x8c\x0b_currencies\x94}\x94(\x8c\x03eur\x94K\x00\x8c\x03usd\x94K\x01u\x8c\x03obj\x94\x8c\x0brateslib.rs\x94\x8c\x07FXRates\x94\x93\x94]\x94\x8c\x0brateslib.rs\x94\x8c\x06FXRate\x94\x93\x94(\x8c\x03eur\x94\x8c\x03usd\x94G?\xf2ffffff\x8c\x08datetime\x94\x8c\x08datetime\x94\x93\x94C\n\x07\xd0\x01\x01\x00\x00\x00\x00\x00\x00\x94\x85\x94R\x94t\x94\x81\x94C>\x03\x00\x00\x00\x00\x00\x00\x00eur\x03\x00\x00\x00\x00\x00\x00\x00usd\x02\x00\x00\x00ffffff\xf2?\x01\x13\x00\x00\x00\x00\x00\x00\x002000-01-01T00:00:00\x94ba\x8c\x0brateslib.rs\x94\x8c\x03Ccy\x94\x93\x94\x8c\x03eur\x94\x85\x94\x81\x94C\x0b\x03\x00\x00\x00\x00\x00\x00\x00eur\x94b\x86\x94\x81\x94Cd\x01\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00eur\x03\x00\x00\x00\x00\x00\x00\x00usd\x02\x00\x00\x00ffffff\xf2?\x01\x13\x00\x00\x00\x00\x00\x00\x002000-01-01T00:00:00\x02\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00eur\x03\x00\x00\x00\x00\x00\x00\x00usd\x94b\x8c\x06_state\x94\x8a\x084x<\x17/\xfc\xc1\x8cub.'
Deserializing
# --------------dual ---------------------
In [51]: loads(dual_bytes)
Out[51]: <Dual: 3.141000, (x, y), [1.0, 0.0]>
In [52]: loads(dual2_bytes)
Out[52]: <Dual2: 3.141000, (x, y), [1.0, 0.0], [[...]]>
In [53]: loads(variable_bytes)
Out[53]: <Variable: 3.141, (z), [1.0]>
# ---------- scheduling ------------------
In [54]: loads(cal_bytes)
Out[54]: <rateslib.rs.Cal at 0x120865080>
In [55]: loads(union_cal_bytes)
Out[55]: <rateslib.rs.UnionCal at 0x12136b460>
In [56]: loads(named_cal_bytes)
Out[56]: <rateslib.rs.NamedCal at 0x121357cf0>
In [57]: loads(adjuster_bytes)
Out[57]: <rl.Adjuster.Following at 0x122179780>
In [58]: loads(imm_bytes)
Out[58]: <rl.Imm.Wed3 at 0x1221799a0>
In [59]: loads(roll_bytes)
Out[59]: <rl.RollDay.Day(31) at 0x122179ac0>
In [60]: loads(frequency_bytes)
Out[60]: <rl.Frequency.Months(4, Day(31)) at 0x121d89b40>
In [61]: loads(stub_inference_bytes)
Out[61]: <rl.StubInference.LongFront at 0x122179960>
In [62]: loads(schedule_bytes)
Out[62]: <rl.Schedule at 0x11fadd4f0>
# ------------ splines --------------------
In [63]: loads(ppspline_f64_bytes)
Out[63]: <rl.PPSplineF64 at 0x120500f90>
In [64]: loads(ppspline_dual_bytes)
Out[64]: <rl.PPSplineDual at 0x120502420>
In [65]: loads(ppspline_dual2_bytes)
Out[65]: <rl.PPSplineDual2 at 0x120502490>
# ------------ fx_rates -------------------
In [66]: loads(fxr_bytes)
Out[66]: <rl.FXRates:[eur,usd] at 0x118fadc50>