Serialization#
Rateslib is an object oriented library requiring dynamic associations to create pricing and risk functionality. The objective of serialization and deserialization is to convert rateslib’s objects into a form that can be persisted or transported, with a view towards being able to recapture those dynamic associations (i.e. saved and loaded from a database, or sent between client and server over a network).
Warning
This feature is currently experimental, and in development. This page documents the objects that have, to date, had focused development for the two forms of serialization rateslib intends to offer. Other objects in rateslib may well have serialization methods in place but the stability and robustness of those may not have been properly tested.
Forms#
Serialization is a general term. The specific forms of serialization that rateslib aims to create is JSON serialization and Python pickling.
JSON serialization converts an object to a string and reconstructs it from that same string. The advantage of JSON is that it is human readable (and often editable) whilst a disadvantage is that it likely to be a lesser efficient form of transfer or storage. It is considered a safe form of serialization.
In [1]: from rateslib import from_json
Create an object from JSON string. |
Pickling is a native conversion of binary objects to byte arrays. That same array can be reconstituted into a Python object. This is typically very efficient but the resultant arrays of data are not human-readable (or editable). This is considered an unsafe form of serialization due to the arbitrary nature of code execution upon deserialization.
Pickling of data is often performed internally by Python’s multiprocessing module if it needs to share data across threads.
In [2]: from pickle import dumps, loads
Objects in Scope#
We aim to keep the objects in scope synchronized with availability for both json and pickle serialization for the stated objects below.
Serializing
# --------------dual ---------------------
In [3]: dual.to_json() # Dual
Out[3]: '{"Dual":{"real":3.141,"vars":["x","y"],"dual":{"v":1,"dim":[2],"data":[1.0,0.0]}}}'
In [4]: dual2.to_json() # Dual2
Out[4]: '{"Dual2":{"real":3.141,"vars":["x","y"],"dual":{"v":1,"dim":[2],"data":[1.0,0.0]},"dual2":{"v":1,"dim":[2,2],"data":[0.0,0.0,0.0,0.0]}}}'
In [5]: variable.to_json() # Variable
Out[5]: '{"PyNative": {"Variable": {"real": 3.141, "vars": ["z"], "dual": [1.0]}}}'
# ---------- scheduling ------------------
In [6]: cal.to_json() # Cal
Out[6]: '{"Cal":{"holidays":[],"week_mask":["Sat","Sun"]}}'
In [7]: union_cal.to_json() # UnionCal
Out[7]: '{"UnionCal":{"calendars":[{"holidays":[],"week_mask":["Sat","Sun"]}],"settlement_calendars":[]}}'
In [8]: named_cal.to_json() # NamedCal
Out[8]: '{"NamedCal":{"name":"bus"}}'
In [9]: adjuster.to_json() # Adjuster
Out[9]: '{"PyAdjuster":{"Following":{"_u8":1}}}'
In [10]: imm.to_json() # Imm
Out[10]: '{"Imm":"Wed3"}'
In [11]: roll.to_json() # RollDay
Out[11]: '{"RollDay":{"Day":31}}'
In [12]: frequency.to_json() # Frequency
Out[12]: '{"Frequency":{"Months":{"number":4,"roll":{"Day":31}}}}'
In [13]: stub_inference.to_json() # StubInference
Out[13]: '{"StubInference":"LongFront"}'
In [14]: schedule.to_json() # Schedule
Out[14]: '{"PyWrapped":{"Schedule":{"ueffective":"2000-01-01T00:00:00","utermination":"2001-01-01T00:00:00","frequency":{"Months":{"number":6,"roll":{"Day":1}}},"ufront_stub":null,"uback_stub":null,"calendar":{"NamedCal":{"name":"all"}},"accrual_adjuster":{"ModifiedFollowing":{}},"payment_adjuster":{"BusDaysLagSettle":2},"payment_adjuster2":{"BusDaysLagSettle":0},"payment_adjuster3":null}}}'
# ------------ splines --------------------
In [15]: ppspline_f64.to_json() # PPSplineF64
Out[15]: '{"PPSplineF64":{"inner":{"k":4,"t":[0.0,0.0,0.0,0.0,4.0,4.0,4.0,4.0],"c":{"v":1,"dim":[4],"data":[0.3,0.2,0.6,0.2]},"n":4}}}'
In [16]: ppspline_dual.to_json() # PPSplineDual
Out[16]: '{"PPSplineDual":{"inner":{"k":3,"t":[0.0,0.0,0.0,1.0,1.0,1.0],"c":{"v":1,"dim":[3],"data":[{"real":0.1,"vars":[],"dual":{"v":1,"dim":[0],"data":[]}},{"real":0.2,"vars":[],"dual":{"v":1,"dim":[0],"data":[]}},{"real":0.3,"vars":[],"dual":{"v":1,"dim":[0],"data":[]}}]},"n":3}}}'
In [17]: ppspline_dual2.to_json() # PPSplineDual2
Out[17]: '{"PPSplineDual2":{"inner":{"k":3,"t":[0.0,0.0,0.0,1.0,1.0,1.0],"c":{"v":1,"dim":[3],"data":[{"real":0.1,"vars":[],"dual":{"v":1,"dim":[0],"data":[]},"dual2":{"v":1,"dim":[0,0],"data":[]}},{"real":0.2,"vars":[],"dual":{"v":1,"dim":[0],"data":[]},"dual2":{"v":1,"dim":[0,0],"data":[]}},{"real":0.3,"vars":[],"dual":{"v":1,"dim":[0],"data":[]},"dual2":{"v":1,"dim":[0,0],"data":[]}}]},"n":3}}}'
# ------------ fx_rates -------------------
In [18]: fxr.to_json() # FXRates
Out[18]: '{"PyWrapped":{"FXRates":{"fx_rates":[{"pair":[{"name":"eur"},{"name":"usd"}],"rate":{"F64":1.15},"settlement":"2000-01-01T00:00:00"}],"currencies":[{"name":"usd"},{"name":"eur"}]}}}'
Deserializing
# --------------dual ---------------------
In [19]: from_json(dual_json)
Out[19]: <Dual: 3.141000, (x, y), [1.0, 0.0]>
In [20]: from_json(dual2_json)
Out[20]: <Dual2: 3.141000, (x, y), [1.0, 0.0], [[...]]>
In [21]: from_json(variable_json)
Out[21]: <Variable: 3.141, (z), [1.0]>
# ---------- scheduling ------------------
In [22]: from_json(cal_json)
Out[22]: <rl.Cal at 0x16b43c0d0>
In [23]: from_json(union_cal_json)
Out[23]: <rl.UnionCal at 0x171e8d530>
In [24]: from_json(named_cal_json)
Out[24]: <rl.NamedCal:'bus' at 0x16ced7d00>
In [25]: from_json(adjuster_json)
Out[25]: <rl.Adjuster.Following at 0x168855fe0>
In [26]: from_json(imm_json)
Out[26]: <rl.Imm.Wed3 at 0x168855f80>
In [27]: from_json(roll_json)
Out[27]: <rl.RollDay.Day(31) at 0x168855a40>
In [28]: from_json(frequency_json)
Out[28]: <rl.Frequency.Months(4, Day(31)) at 0x16b43c310>
In [29]: from_json(stub_inference_json)
Out[29]: <rl.StubInference.LongFront at 0x168856180>
In [30]: from_json(schedule_json)
Out[30]: <rl.Schedule at 0x172ed4100>
# ------------ splines --------------------
In [31]: from_json(ppspline_f64_json)
Out[31]: <rl.PPSplineF64 at 0x169f84a50>
In [32]: from_json(ppspline_dual_json)
Out[32]: <rl.PPSplineDual at 0x169f84b30>
In [33]: from_json(ppspline_dual2_json)
Out[33]: <rl.PPSplineDual2 at 0x169f849e0>
# ------------ fx_rates -------------------
In [34]: from_json(fxr_json)
Out[34]: <rl.FXRates:[usd,eur] at 0x169f84980>
Serializing
# --------------dual ---------------------
In [35]: dumps(dual) # Dual
Out[35]: b'\x80\x05\x95G\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\x04Dual\x94\x93\x94G@\t \xc4\x9b\xa5\xe3T]\x94(\x8c\x01x\x94\x8c\x01y\x94e]\x94(G?\xf0\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00e\x87\x94\x81\x94.'
In [36]: dumps(dual2) # Dual2
Out[36]: b'\x80\x05\x95q\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\x05Dual2\x94\x93\x94(G@\t \xc4\x9b\xa5\xe3T]\x94(\x8c\x01x\x94\x8c\x01y\x94e]\x94(G?\xf0\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00e]\x94(G\x00\x00\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00et\x94\x81\x94.'
In [37]: dumps(variable) # Variable
Out[37]: b'\x80\x05\x95\xce\x00\x00\x00\x00\x00\x00\x00\x8c\x16rateslib.dual.variable\x94\x8c\x08Variable\x94\x93\x94)\x81\x94}\x94(\x8c\x04real\x94G@\t \xc4\x9b\xa5\xe3T\x8c\x04vars\x94\x8c\x01z\x94\x85\x94\x8c\x04dual\x94\x8c\x13numpy._core.numeric\x94\x8c\x0b_frombuffer\x94\x93\x94(\x96\x08\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xf0?\x94\x8c\x05numpy\x94\x8c\x05dtype\x94\x93\x94\x8c\x02f8\x94\x89\x88\x87\x94R\x94(K\x03\x8c\x01<\x94NNNJ\xff\xff\xff\xffJ\xff\xff\xff\xffK\x00t\x94bK\x01\x85\x94\x8c\x01C\x94t\x94R\x94ub.'
# ---------- scheduling ------------------
In [38]: dumps(cal) # Cal
Out[38]: b'\x80\x05\x95"\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\x03Cal\x94\x93\x94]\x94C\x02\x05\x06\x94\x86\x94\x81\x94.'
In [39]: dumps(union_cal) # UnionCal
Out[39]: b'\x80\x05\x95F\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\x08UnionCal\x94\x93\x94]\x94\x8c\x0brateslib.rs\x94\x8c\x03Cal\x94\x93\x94]\x94C\x02\x05\x06\x94\x86\x94\x81\x94a]\x94\x86\x94\x81\x94.'
In [40]: dumps(named_cal) # NamedCal
Out[40]: b'\x80\x05\x95&\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\x08NamedCal\x94\x93\x94\x8c\x03bus\x94\x85\x94\x81\x94.'
In [41]: dumps(adjuster) # Adjuster
Out[41]: b'\x80\x05\x95.\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\x14PyAdjuster_Following\x94\x93\x94K\x01\x85\x94\x81\x94.'
In [42]: dumps(imm) # Imm
Out[42]: b'\x80\x05\x95\x1d\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\x03Imm\x94\x93\x94K\x01\x85\x94\x81\x94.'
In [43]: dumps(roll) # RollDay
Out[43]: b'\x80\x05\x95%\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\x0bRollDay_Day\x94\x93\x94K\x1f\x85\x94\x81\x94.'
In [44]: dumps(frequency) # Frequency
Out[44]: b'\x80\x05\x95N\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\x10Frequency_Months\x94\x93\x94K\x04\x8c\x0brateslib.rs\x94\x8c\x0bRollDay_Day\x94\x93\x94K\x1f\x85\x94\x81\x94\x86\x94\x81\x94.'
In [45]: dumps(stub_inference) # StubInference
Out[45]: b"\x80\x05\x95'\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\rStubInference\x94\x93\x94K\x01\x85\x94\x81\x94."
In [46]: dumps(schedule) # Schedule
Out[46]: b'\x80\x05\x95\xcf\x02\x00\x00\x00\x00\x00\x00\x8c\x1crateslib.scheduling.schedule\x94\x8c\x08Schedule\x94\x93\x94(\x8c\x08datetime\x94\x8c\x08datetime\x94\x93\x94C\n\x07\xd0\x01\x01\x00\x00\x00\x00\x00\x00\x94\x85\x94R\x94h\x05C\n\x07\xd1\x01\x01\x00\x00\x00\x00\x00\x00\x94\x85\x94R\x94\x8c\x0brateslib.rs\x94\x8c\x10Frequency_Months\x94\x93\x94K\x06\x8c\x0brateslib.rs\x94\x8c\x0bRollDay_Day\x94\x93\x94K\x01\x85\x94\x81\x94\x86\x94\x81\x94\x8c\x17rateslib.enums.generics\x94\x8c\x07NoInput\x94\x93\x94K\x00\x85\x94R\x94h\x1ah\x1ah\x1ah\x1a\x8c\x0brateslib.rs\x94\x8c\x1cPyAdjuster_ModifiedFollowing\x94\x93\x94K\x02\x85\x94\x81\x94\x8c\x0brateslib.rs\x94\x8c\x08NamedCal\x94\x93\x94\x8c\x03all\x94\x85\x94\x81\x94\x8c\x0brateslib.rs\x94\x8c\x1bPyAdjuster_BusDaysLagSettle\x94\x93\x94K\x02K\t\x86\x94\x81\x94h(K\x00K\t\x86\x94\x81\x94h\x1ah\x1ah\x1at\x94\x81\x94}\x94(\x8c\x04_obj\x94\x8c\x0brateslib.rs\x94\x8c\x08Schedule\x94\x93\x94(h\x05C\n\x07\xd0\x01\x01\x00\x00\x00\x00\x00\x00\x94\x85\x94R\x94h\x05C\n\x07\xd1\x01\x01\x00\x00\x00\x00\x00\x00\x94\x85\x94R\x94h\x0eK\x06h\x11K\x01\x85\x94\x81\x94\x86\x94\x81\x94h"\x8c\x03all\x94\x85\x94\x81\x94h\x1dK\x02\x85\x94\x81\x94h(K\x02K\t\x86\x94\x81\x94h(K\x00K\t\x86\x94\x81\x94\x89NNNNt\x94\x81\x94\x8c\nueffective\x94h\x08\x8c\x0cutermination\x94h\x0b\x8c\rfrequency_obj\x94h\x15\x8c\x0bufront_stub\x94N\x8c\nuback_stub\x94N\x8c\x10accrual_adjuster\x94h\x1f\x8c\x08calendar\x94h%\x8c\x10payment_adjuster\x94h*\x8c\x11payment_adjuster2\x94h,\x8c\x11payment_adjuster3\x94Nub.'
# ------------ splines --------------------
In [47]: dumps(ppspline_f64) # PPSplineF64
Out[47]: b'\x80\x05\x95\x99\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\x0bPPSplineF64\x94\x93\x94K\x04]\x94(G\x00\x00\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00G@\x10\x00\x00\x00\x00\x00\x00G@\x10\x00\x00\x00\x00\x00\x00G@\x10\x00\x00\x00\x00\x00\x00G@\x10\x00\x00\x00\x00\x00\x00e]\x94(G?\xd3333333G?\xc9\x99\x99\x99\x99\x99\x9aG?\xe3333333G?\xc9\x99\x99\x99\x99\x99\x9ae\x87\x94\x81\x94.'
In [48]: dumps(ppspline_dual) # PPSplineDual
Out[48]: b'\x80\x05\x95\xb2\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\x0cPPSplineDual\x94\x93\x94K\x03]\x94(G\x00\x00\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00G?\xf0\x00\x00\x00\x00\x00\x00G?\xf0\x00\x00\x00\x00\x00\x00G?\xf0\x00\x00\x00\x00\x00\x00e]\x94(\x8c\x0brateslib.rs\x94\x8c\x04Dual\x94\x93\x94G?\xb9\x99\x99\x99\x99\x99\x9a]\x94]\x94\x87\x94\x81\x94h\x07G?\xc9\x99\x99\x99\x99\x99\x9a]\x94]\x94\x87\x94\x81\x94h\x07G?\xd3333333]\x94]\x94\x87\x94\x81\x94e\x87\x94\x81\x94.'
In [49]: dumps(ppspline_dual2) # PPSplineDual2
Out[49]: b'\x80\x05\x95\xbd\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\rPPSplineDual2\x94\x93\x94K\x03]\x94(G\x00\x00\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00G?\xf0\x00\x00\x00\x00\x00\x00G?\xf0\x00\x00\x00\x00\x00\x00G?\xf0\x00\x00\x00\x00\x00\x00e]\x94(\x8c\x0brateslib.rs\x94\x8c\x05Dual2\x94\x93\x94(G?\xb9\x99\x99\x99\x99\x99\x9a]\x94]\x94]\x94t\x94\x81\x94h\x07(G?\xc9\x99\x99\x99\x99\x99\x9a]\x94]\x94]\x94t\x94\x81\x94h\x07(G?\xd3333333]\x94]\x94]\x94t\x94\x81\x94e\x87\x94\x81\x94.'
# ------------ fx_rates -------------------
In [50]: dumps(fxr) # FXRates
Out[50]: b'\x80\x05\x95\xbd\x01\x00\x00\x00\x00\x00\x00\x8c\x14rateslib.fx.fx_rates\x94\x8c\x07FXRates\x94\x93\x94)\x81\x94}\x94(\x8c\x0b_currencies\x94}\x94(\x8c\x03usd\x94K\x00\x8c\x03eur\x94K\x01u\x8c\x03obj\x94\x8c\x0brateslib.rs\x94\x8c\x07FXRates\x94\x93\x94]\x94\x8c\x0brateslib.rs\x94\x8c\x06FXRate\x94\x93\x94(\x8c\x03eur\x94\x8c\x03usd\x94G?\xf2ffffff\x8c\x08datetime\x94\x8c\x08datetime\x94\x93\x94C\n\x07\xd0\x01\x01\x00\x00\x00\x00\x00\x00\x94\x85\x94R\x94t\x94\x81\x94C>\x03\x00\x00\x00\x00\x00\x00\x00eur\x03\x00\x00\x00\x00\x00\x00\x00usd\x02\x00\x00\x00ffffff\xf2?\x01\x13\x00\x00\x00\x00\x00\x00\x002000-01-01T00:00:00\x94ba\x8c\x0brateslib.rs\x94\x8c\x03Ccy\x94\x93\x94\x8c\x03usd\x94\x85\x94\x81\x94C\x0b\x03\x00\x00\x00\x00\x00\x00\x00usd\x94b\x86\x94\x81\x94Cd\x01\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00eur\x03\x00\x00\x00\x00\x00\x00\x00usd\x02\x00\x00\x00ffffff\xf2?\x01\x13\x00\x00\x00\x00\x00\x00\x002000-01-01T00:00:00\x02\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00usd\x03\x00\x00\x00\x00\x00\x00\x00eur\x94b\x8c\x06_state\x94\x8a\x08{\xbf\xe9\x9c"\xd3\xc3\xb3ub.'
Deserializing
# --------------dual ---------------------
In [51]: loads(dual_bytes)
Out[51]: <Dual: 3.141000, (x, y), [1.0, 0.0]>
In [52]: loads(dual2_bytes)
Out[52]: <Dual2: 3.141000, (x, y), [1.0, 0.0], [[...]]>
In [53]: loads(variable_bytes)
Out[53]: <Variable: 3.141, (z), [1.0]>
# ---------- scheduling ------------------
In [54]: loads(cal_bytes)
Out[54]: <rl.Cal at 0x142167760>
In [55]: loads(union_cal_bytes)
Out[55]: <rl.UnionCal at 0x171eef4c0>
In [56]: loads(named_cal_bytes)
Out[56]: <rl.NamedCal:'bus' at 0x16d24d960>
In [57]: loads(adjuster_bytes)
Out[57]: <rl.Adjuster.Following at 0x168857da0>
In [58]: loads(imm_bytes)
Out[58]: <rl.Imm.Wed3 at 0x168857ea0>
In [59]: loads(roll_bytes)
Out[59]: <rl.RollDay.Day(31) at 0x168857f20>
In [60]: loads(frequency_bytes)
Out[60]: <rl.Frequency.Months(4, Day(31)) at 0x170ebe0e0>
In [61]: loads(stub_inference_bytes)
Out[61]: <rl.StubInference.LongFront at 0x16d0fe640>
In [62]: loads(schedule_bytes)
Out[62]: <rl.Schedule at 0x171d64c00>
# ------------ splines --------------------
In [63]: loads(ppspline_f64_bytes)
Out[63]: <rl.PPSplineF64 at 0x1615bae30>
In [64]: loads(ppspline_dual_bytes)
Out[64]: <rl.PPSplineDual at 0x1615bb140>
In [65]: loads(ppspline_dual2_bytes)
Out[65]: <rl.PPSplineDual2 at 0x1615bb1b0>
# ------------ fx_rates -------------------
In [66]: loads(fxr_bytes)
Out[66]: <rl.FXRates:[usd,eur] at 0x1615bacf0>