Serialization#
Rateslib is an object oriented library requiring dynamic associations to create pricing and risk functionality. The objective of serialization and deserialization is to convert rateslib’s objects into a form that can be persisted or transported, with a view towards being able to recapture those dynamic associations (i.e. saved and loaded from a database, or sent between client and server over a network).
Warning
This feature is currently experimental, and in development. This page documents the objects that have, to date, had focused development for the two forms of serialization rateslib intends to offer. Other objects in rateslib may well have serialization methods in place but the stability and robustness of those may not have been properly tested.
Forms#
Serialization is a general term. The specific forms of serialization that rateslib aims to create is JSON serialization and Python pickling.
JSON serialization converts an object to a string and reconstructs it from that same string. The advantage of JSON is that it is human readable (and often editable) whilst a disadvantage is that it likely to be a lesser efficient form of transfer or storage. It is considered a safe form of serialization.
In [1]: from rateslib import from_json
Create an object from JSON string. |
Pickling is a native conversion of binary objects to byte arrays. That same array can be reconstituted into a Python object. This is typically very efficient but the resultant arrays of data are not human-readable (or editable). This is considered an unsafe form of serialization due to the arbitrary nature of code execution upon deserialization.
Pickling of data is often performed internally by Python’s multiprocessing module if it needs to share data across threads.
In [2]: from pickle import dumps, loads
Objects in Scope#
We aim to keep the objects in scope synchronized with availability for both json and pickle serialization for the stated objects below.
Serializing
# --------------dual ---------------------
In [3]: dual.to_json() # Dual
Out[3]: '{"Dual":{"real":3.141,"vars":["x","y"],"dual":{"v":1,"dim":[2],"data":[1.0,0.0]}}}'
In [4]: dual2.to_json() # Dual2
Out[4]: '{"Dual2":{"real":3.141,"vars":["x","y"],"dual":{"v":1,"dim":[2],"data":[1.0,0.0]},"dual2":{"v":1,"dim":[2,2],"data":[0.0,0.0,0.0,0.0]}}}'
In [5]: variable.to_json() # Variable
Out[5]: '{"PyNative": {"Variable": {"real": 3.141, "vars": ["z"], "dual": [1.0]}}}'
# ---------- scheduling ------------------
In [6]: cal.to_json() # Cal
Out[6]: '{"Cal":{"holidays":[],"week_mask":["Sun","Sat"]}}'
In [7]: union_cal.to_json() # UnionCal
Out[7]: '{"UnionCal":{"calendars":[{"holidays":[],"week_mask":["Sun","Sat"]}],"settlement_calendars":[]}}'
In [8]: named_cal.to_json() # NamedCal
Out[8]: '{"NamedCal":{"name":"bus"}}'
In [9]: adjuster.to_json() # Adjuster
Out[9]: '{"PyAdjuster":{"Following":{"_u8":1}}}'
In [10]: imm.to_json() # Imm
Out[10]: '{"Imm":"Wed3"}'
In [11]: roll.to_json() # RollDay
Out[11]: '{"RollDay":{"Day":31}}'
In [12]: frequency.to_json() # Frequency
Out[12]: '{"Frequency":{"Months":{"number":4,"roll":{"Day":31}}}}'
In [13]: stub_inference.to_json() # StubInference
Out[13]: '{"StubInference":"LongFront"}'
In [14]: schedule.to_json() # Schedule
Out[14]: '{"PyWrapped":{"Schedule":{"ueffective":"2000-01-01T00:00:00","utermination":"2001-01-01T00:00:00","frequency":{"Months":{"number":6,"roll":{"Day":1}}},"ufront_stub":null,"uback_stub":null,"calendar":{"NamedCal":{"name":"all"}},"accrual_adjuster":{"ModifiedFollowing":{}},"payment_adjuster":{"BusDaysLagSettle":2}}}}'
# ------------ splines --------------------
In [15]: ppspline_f64.to_json() # PPSplineF64
Out[15]: '{"PPSplineF64":{"inner":{"k":4,"t":[0.0,0.0,0.0,0.0,4.0,4.0,4.0,4.0],"c":{"v":1,"dim":[4],"data":[0.3,0.2,0.6,0.2]},"n":4}}}'
In [16]: ppspline_dual.to_json() # PPSplineDual
Out[16]: '{"PPSplineDual":{"inner":{"k":3,"t":[0.0,0.0,0.0,1.0,1.0,1.0],"c":{"v":1,"dim":[3],"data":[{"real":0.1,"vars":[],"dual":{"v":1,"dim":[0],"data":[]}},{"real":0.2,"vars":[],"dual":{"v":1,"dim":[0],"data":[]}},{"real":0.3,"vars":[],"dual":{"v":1,"dim":[0],"data":[]}}]},"n":3}}}'
In [17]: ppspline_dual2.to_json() # PPSplineDual2
Out[17]: '{"PPSplineDual2":{"inner":{"k":3,"t":[0.0,0.0,0.0,1.0,1.0,1.0],"c":{"v":1,"dim":[3],"data":[{"real":0.1,"vars":[],"dual":{"v":1,"dim":[0],"data":[]},"dual2":{"v":1,"dim":[0,0],"data":[]}},{"real":0.2,"vars":[],"dual":{"v":1,"dim":[0],"data":[]},"dual2":{"v":1,"dim":[0,0],"data":[]}},{"real":0.3,"vars":[],"dual":{"v":1,"dim":[0],"data":[]},"dual2":{"v":1,"dim":[0,0],"data":[]}}]},"n":3}}}'
# ------------ fx_rates -------------------
In [18]: fxr.to_json() # FXRates
Out[18]: '{"PyWrapped":{"FXRates":{"fx_rates":[{"pair":[{"name":"eur"},{"name":"usd"}],"rate":{"F64":1.15},"settlement":"2000-01-01T00:00:00"}],"currencies":[{"name":"eur"},{"name":"usd"}]}}}'
Deserializing
# --------------dual ---------------------
In [19]: from_json(dual_json)
Out[19]: <Dual: 3.141000, (x, y), [1.0, 0.0]>
In [20]: from_json(dual2_json)
Out[20]: <Dual2: 3.141000, (x, y), [1.0, 0.0], [[...]]>
In [21]: from_json(variable_json)
Out[21]: <Variable: 3.141, (z), [1.0]>
# ---------- scheduling ------------------
In [22]: from_json(cal_json)
Out[22]: <rateslib.rs.Cal at 0x122ce6700>
In [23]: from_json(union_cal_json)
Out[23]: <rateslib.rs.UnionCal at 0x122d72510>
In [24]: from_json(named_cal_json)
Out[24]: <rateslib.rs.NamedCal at 0x122d6c030>
In [25]: from_json(adjuster_json)
Out[25]: <rl.Adjuster.Following at 0x122d2dcc0>
In [26]: from_json(imm_json)
Out[26]: <rl.Imm.Wed3 at 0x122d2dd00>
In [27]: from_json(roll_json)
Out[27]: <rl.RollDay.Day(31) at 0x122d2dd60>
In [28]: from_json(frequency_json)
Out[28]: <rl.Frequency.Months(4, Day(31)) at 0x122ce6200>
In [29]: from_json(stub_inference_json)
Out[29]: <rl.StubInference.LongFront at 0x122d2de20>
In [30]: from_json(schedule_json)
Out[30]: <rl.Schedule at 0x120eabb90>
# ------------ splines --------------------
In [31]: from_json(ppspline_f64_json)
Out[31]: <rl.PPSplineF64 at 0x122d68740>
In [32]: from_json(ppspline_dual_json)
Out[32]: <rl.PPSplineDual at 0x122d68a50>
In [33]: from_json(ppspline_dual2_json)
Out[33]: <rl.PPSplineDual2 at 0x122d68ba0>
# ------------ fx_rates -------------------
In [34]: from_json(fxr_json)
Out[34]: <rl.FXRates:[eur,usd] at 0x122d688a0>
Serializing
# --------------dual ---------------------
In [35]: dumps(dual) # Dual
Out[35]: b'\x80\x04\x95G\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\x04Dual\x94\x93\x94G@\t \xc4\x9b\xa5\xe3T]\x94(\x8c\x01x\x94\x8c\x01y\x94e]\x94(G?\xf0\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00e\x87\x94\x81\x94.'
In [36]: dumps(dual2) # Dual2
Out[36]: b'\x80\x04\x95q\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\x05Dual2\x94\x93\x94(G@\t \xc4\x9b\xa5\xe3T]\x94(\x8c\x01x\x94\x8c\x01y\x94e]\x94(G?\xf0\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00e]\x94(G\x00\x00\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00et\x94\x81\x94.'
In [37]: dumps(variable) # Variable
Out[37]: b'\x80\x04\x95\xe3\x00\x00\x00\x00\x00\x00\x00\x8c\x16rateslib.dual.variable\x94\x8c\x08Variable\x94\x93\x94)\x81\x94}\x94(\x8c\x04real\x94G@\t \xc4\x9b\xa5\xe3T\x8c\x04vars\x94\x8c\x01z\x94\x85\x94\x8c\x04dual\x94\x8c\x16numpy._core.multiarray\x94\x8c\x0c_reconstruct\x94\x93\x94\x8c\x05numpy\x94\x8c\x07ndarray\x94\x93\x94K\x00\x85\x94C\x01b\x94\x87\x94R\x94(K\x01K\x01\x85\x94h\r\x8c\x05dtype\x94\x93\x94\x8c\x02f8\x94\x89\x88\x87\x94R\x94(K\x03\x8c\x01<\x94NNNJ\xff\xff\xff\xffJ\xff\xff\xff\xffK\x00t\x94b\x89C\x08\x00\x00\x00\x00\x00\x00\xf0?\x94t\x94bub.'
# ---------- scheduling ------------------
In [38]: dumps(cal) # Cal
Out[38]: b'\x80\x04\x95"\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\x03Cal\x94\x93\x94]\x94C\x02\x06\x05\x94\x86\x94\x81\x94.'
In [39]: dumps(union_cal) # UnionCal
Out[39]: b'\x80\x04\x95F\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\x08UnionCal\x94\x93\x94]\x94\x8c\x0brateslib.rs\x94\x8c\x03Cal\x94\x93\x94]\x94C\x02\x06\x05\x94\x86\x94\x81\x94a]\x94\x86\x94\x81\x94.'
In [40]: dumps(named_cal) # NamedCal
Out[40]: b'\x80\x04\x95&\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\x08NamedCal\x94\x93\x94\x8c\x03bus\x94\x85\x94\x81\x94.'
In [41]: dumps(adjuster) # Adjuster
Out[41]: b'\x80\x04\x95.\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\x14PyAdjuster_Following\x94\x93\x94K\x01\x85\x94\x81\x94.'
In [42]: dumps(imm) # Imm
Out[42]: b'\x80\x04\x95\x1d\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\x03Imm\x94\x93\x94K\x01\x85\x94\x81\x94.'
In [43]: dumps(roll) # RollDay
Out[43]: b'\x80\x04\x95%\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\x0bRollDay_Day\x94\x93\x94K\x1f\x85\x94\x81\x94.'
In [44]: dumps(frequency) # Frequency
Out[44]: b'\x80\x04\x95N\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\x10Frequency_Months\x94\x93\x94K\x04\x8c\x0brateslib.rs\x94\x8c\x0bRollDay_Day\x94\x93\x94K\x1f\x85\x94\x81\x94\x86\x94\x81\x94.'
In [45]: dumps(stub_inference) # StubInference
Out[45]: b"\x80\x04\x95'\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\rStubInference\x94\x93\x94K\x01\x85\x94\x81\x94."
In [46]: dumps(schedule) # Schedule
Out[46]: b'\x80\x04\x95\x86\x02\x00\x00\x00\x00\x00\x00\x8c\x1crateslib.scheduling.schedule\x94\x8c\x08Schedule\x94\x93\x94(\x8c\x08datetime\x94\x8c\x08datetime\x94\x93\x94C\n\x07\xd0\x01\x01\x00\x00\x00\x00\x00\x00\x94\x85\x94R\x94h\x05C\n\x07\xd1\x01\x01\x00\x00\x00\x00\x00\x00\x94\x85\x94R\x94\x8c\x0brateslib.rs\x94\x8c\x10Frequency_Months\x94\x93\x94K\x06\x8c\x0brateslib.rs\x94\x8c\x0bRollDay_Day\x94\x93\x94K\x01\x85\x94\x81\x94\x86\x94\x81\x94\x8c\x10rateslib.default\x94\x8c\x07NoInput\x94\x93\x94K\x00\x85\x94R\x94h\x1ah\x1ah\x1ah\x1a\x8c\x0brateslib.rs\x94\x8c\x1cPyAdjuster_ModifiedFollowing\x94\x93\x94K\x02\x85\x94\x81\x94\x8c\x0brateslib.rs\x94\x8c\x08NamedCal\x94\x93\x94\x8c\x03all\x94\x85\x94\x81\x94\x8c\x0brateslib.rs\x94\x8c\x1bPyAdjuster_BusDaysLagSettle\x94\x93\x94K\x02K\t\x86\x94\x81\x94h\x1ah\x1at\x94\x81\x94}\x94(\x8c\x04_obj\x94\x8c\x0brateslib.rs\x94\x8c\x08Schedule\x94\x93\x94(h\x05C\n\x07\xd0\x01\x01\x00\x00\x00\x00\x00\x00\x94\x85\x94R\x94h\x05C\n\x07\xd1\x01\x01\x00\x00\x00\x00\x00\x00\x94\x85\x94R\x94h\x0eK\x06h\x11K\x01\x85\x94\x81\x94\x86\x94\x81\x94h"\x8c\x03all\x94\x85\x94\x81\x94h\x1dK\x02\x85\x94\x81\x94h(K\x02K\t\x86\x94\x81\x94\x89NNNt\x94\x81\x94\x8c\nueffective\x94h\x08\x8c\x0cutermination\x94h\x0b\x8c\rfrequency_obj\x94h\x15\x8c\x0bufront_stub\x94N\x8c\nuback_stub\x94N\x8c\x10accrual_adjuster\x94h\x1f\x8c\x08calendar\x94h%\x8c\x10payment_adjuster\x94h*ub.'
# ------------ splines --------------------
In [47]: dumps(ppspline_f64) # PPSplineF64
Out[47]: b'\x80\x04\x95\x99\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\x0bPPSplineF64\x94\x93\x94K\x04]\x94(G\x00\x00\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00G@\x10\x00\x00\x00\x00\x00\x00G@\x10\x00\x00\x00\x00\x00\x00G@\x10\x00\x00\x00\x00\x00\x00G@\x10\x00\x00\x00\x00\x00\x00e]\x94(G?\xd3333333G?\xc9\x99\x99\x99\x99\x99\x9aG?\xe3333333G?\xc9\x99\x99\x99\x99\x99\x9ae\x87\x94\x81\x94.'
In [48]: dumps(ppspline_dual) # PPSplineDual
Out[48]: b'\x80\x04\x95\xb2\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\x0cPPSplineDual\x94\x93\x94K\x03]\x94(G\x00\x00\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00G?\xf0\x00\x00\x00\x00\x00\x00G?\xf0\x00\x00\x00\x00\x00\x00G?\xf0\x00\x00\x00\x00\x00\x00e]\x94(\x8c\x0brateslib.rs\x94\x8c\x04Dual\x94\x93\x94G?\xb9\x99\x99\x99\x99\x99\x9a]\x94]\x94\x87\x94\x81\x94h\x07G?\xc9\x99\x99\x99\x99\x99\x9a]\x94]\x94\x87\x94\x81\x94h\x07G?\xd3333333]\x94]\x94\x87\x94\x81\x94e\x87\x94\x81\x94.'
In [49]: dumps(ppspline_dual2) # PPSplineDual2
Out[49]: b'\x80\x04\x95\xbd\x00\x00\x00\x00\x00\x00\x00\x8c\x0brateslib.rs\x94\x8c\rPPSplineDual2\x94\x93\x94K\x03]\x94(G\x00\x00\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00G\x00\x00\x00\x00\x00\x00\x00\x00G?\xf0\x00\x00\x00\x00\x00\x00G?\xf0\x00\x00\x00\x00\x00\x00G?\xf0\x00\x00\x00\x00\x00\x00e]\x94(\x8c\x0brateslib.rs\x94\x8c\x05Dual2\x94\x93\x94(G?\xb9\x99\x99\x99\x99\x99\x9a]\x94]\x94]\x94t\x94\x81\x94h\x07(G?\xc9\x99\x99\x99\x99\x99\x9a]\x94]\x94]\x94t\x94\x81\x94h\x07(G?\xd3333333]\x94]\x94]\x94t\x94\x81\x94e\x87\x94\x81\x94.'
# ------------ fx_rates -------------------
In [50]: dumps(fxr) # FXRates
Out[50]: b'\x80\x04\x95\xbd\x01\x00\x00\x00\x00\x00\x00\x8c\x14rateslib.fx.fx_rates\x94\x8c\x07FXRates\x94\x93\x94)\x81\x94}\x94(\x8c\x0b_currencies\x94}\x94(\x8c\x03eur\x94K\x00\x8c\x03usd\x94K\x01u\x8c\x03obj\x94\x8c\x0brateslib.rs\x94\x8c\x07FXRates\x94\x93\x94]\x94\x8c\x0brateslib.rs\x94\x8c\x06FXRate\x94\x93\x94(\x8c\x03eur\x94\x8c\x03usd\x94G?\xf2ffffff\x8c\x08datetime\x94\x8c\x08datetime\x94\x93\x94C\n\x07\xd0\x01\x01\x00\x00\x00\x00\x00\x00\x94\x85\x94R\x94t\x94\x81\x94C>\x03\x00\x00\x00\x00\x00\x00\x00eur\x03\x00\x00\x00\x00\x00\x00\x00usd\x02\x00\x00\x00ffffff\xf2?\x01\x13\x00\x00\x00\x00\x00\x00\x002000-01-01T00:00:00\x94ba\x8c\x0brateslib.rs\x94\x8c\x03Ccy\x94\x93\x94\x8c\x03eur\x94\x85\x94\x81\x94C\x0b\x03\x00\x00\x00\x00\x00\x00\x00eur\x94b\x86\x94\x81\x94Cd\x01\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00eur\x03\x00\x00\x00\x00\x00\x00\x00usd\x02\x00\x00\x00ffffff\xf2?\x01\x13\x00\x00\x00\x00\x00\x00\x002000-01-01T00:00:00\x02\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00eur\x03\x00\x00\x00\x00\x00\x00\x00usd\x94b\x8c\x06_state\x94\x8a\x08\x99B\x00\xea\x1e\x1f\x04\xffub.'
Deserializing
# --------------dual ---------------------
In [51]: loads(dual_bytes)
Out[51]: <Dual: 3.141000, (x, y), [1.0, 0.0]>
In [52]: loads(dual2_bytes)
Out[52]: <Dual2: 3.141000, (x, y), [1.0, 0.0], [[...]]>
In [53]: loads(variable_bytes)
Out[53]: <Variable: 3.141, (z), [1.0]>
# ---------- scheduling ------------------
In [54]: loads(cal_bytes)
Out[54]: <rateslib.rs.Cal at 0x122ce4810>
In [55]: loads(union_cal_bytes)
Out[55]: <rateslib.rs.UnionCal at 0x1220e6f60>
In [56]: loads(named_cal_bytes)
Out[56]: <rateslib.rs.NamedCal at 0x1220c5470>
In [57]: loads(adjuster_bytes)
Out[57]: <rl.Adjuster.Following at 0x122d2e9c0>
In [58]: loads(imm_bytes)
Out[58]: <rl.Imm.Wed3 at 0x122d2ea60>
In [59]: loads(roll_bytes)
Out[59]: <rl.RollDay.Day(31) at 0x122d2eac0>
In [60]: loads(frequency_bytes)
Out[60]: <rl.Frequency.Months(4, Day(31)) at 0x122ce68c0>
In [61]: loads(stub_inference_bytes)
Out[61]: <rl.StubInference.LongFront at 0x122d2ea80>
In [62]: loads(schedule_bytes)
Out[62]: <rl.Schedule at 0x120cc50d0>
# ------------ splines --------------------
In [63]: loads(ppspline_f64_bytes)
Out[63]: <rl.PPSplineF64 at 0x1212ec190>
In [64]: loads(ppspline_dual_bytes)
Out[64]: <rl.PPSplineDual at 0x1212eda80>
In [65]: loads(ppspline_dual2_bytes)
Out[65]: <rl.PPSplineDual2 at 0x1212ec270>
# ------------ fx_rates -------------------
In [66]: loads(fxr_bytes)
Out[66]: <rl.FXRates:[eur,usd] at 0x119c19d30>