Algorithmic trading has revolutionized financial markets, enabling traders to execute high-speed, data-driven decisions. Whether you’re a quantitative trader or an enthusiast exploring crypto trading strategies, Python offers powerful tools to automate trades. In this guide, we’ll explore algorithmic trading with Python, discuss the best algorithmic trading software in the USA, and dive into quantitative analysis for trading in Singapore. We’ll also cover Freqtrade strategy, a popular open-source framework for developing custom trading bots.
We have file lines inside the file very big file so let’s say your RAM is 8 GB but the file is of 80GB so if you try to load without any parameters it will crash your system so this is how you can use this and it’s really amazing it’s really fantastic so flexible but so powerful so now you just imagine that you have coded a file and you made that file binary and you transferred that binary file to some other system some other server and there you are able to use the same class which you have written here and that is the real use case of this pickle file welcome back to the day 16 of the 100 days of hell with python algo trading so in previous session we have learned about file handling we have seen the type of files and also the type of modes we use in the file handling like read mode write mode append mode and we have seen some use cases so today we will continue with that only and we will see what are the problems in the text mode and how we can improve that and also we’ll understand one of the most important topic of the file handling which is serialization and deserialization and then we will understand pickling with the help of pickling we can convert any python object into a bite stream or we can say a binary file and.
We will also understand the unpickling because these are one of the most most important topic of the Python file handling because we’ll always be using the pickling and unpickling so let’s get started with today’s topic so previously we have seen that how we can use contact manager to handle the different kind of files right and we have also seen that we have used to read and write to the files so let me quickly show it again so let’s say what we can do let’s create a new file how we can achieve that we have to use the with keyword so with open and here we’ll give the file name so trades do let’s say txt and here if the file is not existing we have to type w for the right mode and then we can give the file name so let’s say as F do right now you can give your text you want to write right this session is for file handling and when I hit shift enter here you can see that in day 16 there is no file with the name trades. txt so when I had shift enter we have new file here with the name trade. txt and when I click on that we have the data inside that right now let’s say we want to write multiple lines so what we can do we can just copy it paste here uh multiple times and we know that we want all these lines in new lines so for that what we can do we can write sln and here also sln and to differentiate between the lines I’ll just give one two here it could be three four and five right so when we hit shift enter and check the trade Z dxt we have file lines inside the file so now let’s do one more thing that in the next line what I’ll do I’ll try to read this file with the context manager so what is happening here this context manager first open the file and then in the buffer.
We can perform our operations it could be a read operation it could be write operation right and after that it automatically close the file right so let’s say I want to read this file so what I’ll do I’ll just write the same withd and same the open and the file name so it’s straight. txt then here instead of writing I’ll give a read mode as F and now I can just give read and now when I hit shift enter it doesn’t display anything because we have to print this so I’ll just print this and now we can see the whole content but the issue we have discussed already that it loads the whole file and sometime we don’t want that so what we can do for that we can just type the number of characters we want and now when I hit shift enter it will just display the 10 characters so this is another smart method with the help of that we can read a very big file so let’s say your RAM is 8 GB but the file is of 80GB so if you try to load without any parameters it will crash your system right so instead of that what you can do you can load that file in chunks right so what you’ll do you will just write 10 10 so the buffer will load the file in the chunk of 10 characters so now when I shift enter it is loading 10 characters first then the next 10 so here you might have one question that when you write this code so why it is not starting from the initial position right ideally it should be happen but the buffer remembers your position so.
It will load from the the next index so let’s say after 10 it load from 11 then again 21 and so on and so forth okay one more thing I’d like to explain you here that there are two functions one is tell and another is seek so with the help of tell you can know your current position so let’s say if I write here print F do tell and when you hit shift enter it will show you that currently you are at the 10th position and again let’s say if I write it here and again here so you can validate your function right hit shift enter it will show us that first 10 20 30 40 with the help of tell you can know your current position so with the help of seek function you can navigate in your file anywhere so let’s take an example so while you play any video on YouTube you can see a a cursor in below and you can move that anywhere so if you want move from the 1 minute to 5 minute you can just pick that cursor or we can say pointer to 5 minutes so it will just take you there and you call that operation as seek so you are just seeking from the one minute position to 5 minute position right similarly here also you can move from one position to another position so let’s say what I’ll do here in the next line we’ll write this then again what we can do we can print F do read and we can load the whole file for now okay now let’s check the seek function so what I’ll do F do seek I want to go to the 10th position when I had shift and it doesn’t show anything I want to go to the 20th position and now I want to check that where I am right so first we’ll check with that tell function that where we are so print and tell and now after seeking also.
We will check our position when I hit shift enter uh it shows me that okay so what we can do actually I’ll just remove first add 10 here and now when I hit shift enter you can see that first it printed 10 characters then it is showing us our current position then it will take us to the 20th position and after that if I try to again print the 10 characters what it will do it will print the next 10 characters and again you can print the next 10 so you can see that how we can use the seek and tell function so this is quite useful sometimes and have a understanding of this this was the seek during a read operation and now let’s say I want to use seek dur during the right operations so let’s take another cell and inside that what we will do we’ll write something so so first I’ll write uh F do and here let’s say I want to write testing seek function and now when I hit shift enter and check the trades. txt file it will show us the testing seek function now let’s say I will write another line here this is second line and I’ll just change it to like I’ll add one and here the next line so slash n and two dot so we’ll get two lines right so when I head shift enter let’s use the seek function here I’ll use seek and I’ll give let’s say zero now I will write the second line so what happens if I write without this I’ll show you again we will get two lines and you can see here now let’s use the seek function here so what we’ll do I’ll add seek I’ll just mention the zero here it will write this cont content the first line testing seek function then what it will do currently the cursor is at this position right let’s say inside the buffer but this SE is forcing it back to go here at this line one right and now.
It will write this line again what I’ll do here I just remove this slash n and now when I hit shift enter you will see that this line will be replaced by this F correct so let’s check this and it is done so that is the real use of seek function like you can test this as per your requirement and and let us know if you have any doubt in that now let’s take another example of the seek so what I’ll do here instead of zero I’ll add 10 what it means first it will write this line and then the SE function will take me to the 10th position and then it will write this line correct so let’s hit shift enter and check the file again so you’ll see that first it return the whole line then again the seek function take the cursor to this 10 position and then it will write again the second line so this is how the seek function works you can use it as per your requirement and let us know if you face any issue now the issue is that with this kind of operation we can only deal with this string data type with the other data types we cannot use that now here we have two issues first with this read and write mode we cannot work with the binary file the first second is that it is only suitable for the string data type we cannot use tle dictionary here so that’s a drawback of this and in order to resolve that issue what we we can do we have to tweak this little bit and let’s do that what we will do here let me take the screenshot of uh something so what I’ll do I’ll quickly transfer image that is binary file right so I’ll just quickly take a screenshot here you get the idea right this is a screenshot and this is a binary file let’s go back to the code we’ll try to operate uh open and I’ll give the file name that is screenshot yes PNG and and okay now I’ll add again with open here I’ll add the name of the screenshot PNG open in the read mode as let’s say F do read we can print this right generally we print the read function I’ll print and H shift and enter and it says it says UTF 8 codc cannot decode bite 0x89 in position 01 invalid start bite so that means with the help of these simple read we cannot open the binary files so now what we can do to open this file for that what we can do.
We have to use another mode so I’ll just copy and paste it on the next line add a new line of code and here instead of read I’ll write here read binary and I’ll remove this for now and while reading what we have to do we have to write also so with open and now I’ll give a new name to this so it will be completely new file so I’ll write here new screenshot Dot and here we are writing right because we are creating a new file so read binary and we can give here like as G it doesn’t matter right then what we can do first we have opened this read binary right so we have to read this file so I’ll write F do read and simultaneously we will write on this also so I will give here G do WR hope you’re getting my point first we have opened this file in the read binary mode we are reading this file and simultaneously we are writing also right so now when I hit shift enter we can see that we have a new file with the name new screenshot and when we open this we have exactly same screenshot so this is how you can work with the binary files correct hope you are getting my point and actually you just have to practice this and in a very short span of time you’ll be very good at this so just give your enough time to practicing python ultimately it will help you a lot in algor traing so first we have resolved our first problem that uh we cannot work with the binary files so with the help of this read binary and write binary mode we can perform operations on the binary files now the second problem with this text mode was that we cannot write any other data type except string and let me show you how for example uh let’s take this file and trade. txt so what we’ll do we’ll write with open and trra dot txt let’s say in write mode so then what we’ll do we’ll try to write so okay here we have to add as F and let’s write F do right and
Let’s say I’m writing five as an integer right when I hit shift enter it shows me type error which says right argument must be string not int so forcefully I have to make this integer as string and when I run this it works fine and let’s check this file it says five and actually this is not a good way this is really problematic let’s say we already have this five inside and if I want to let’s say perform some action so five and let’s take in this read mode let’s say I want to add another integer with this so when I hit shift enter and it again says can only concatenate string not integer so that’s the big problem so let’s try with addictionary with the name d is equals to let’s say name is equals toip and we can say and let’s say City Singapore and we can also write country I can say bat in bracket you can write India optional so seems good okay so when I hit showing me error invalid syntax we forgot to add comma here okay let’s try to add this dictionary to the file so what I’ll do I’ll open file name uh trade do txt and write mode as F and then what I’ll do I’ll write F dot right I’ll just pass this dictionary so D And now when I hit shift enter it says same arguments must be string not dictionary so that’s the problem so for that so to write this dictionary what I need to do I need to forcefully change my dictionary to a string so I’ll change it to a string and now when I write it works and we can even check this let me print it out so I’ll and read and I’ll just remove this not required and we can print this so print and what it says arguments must be okay uh we have to remove this D not required yeah now.
We can see that we have a dictionary now we start with the most important topic of the day that is serialization and deserialization and pickling these three topics are really very very important and you will be always using them and now let me tell you so what is the purpose and use case of serialization and deserialization so with the help of this we can convert any object to a form in which we can easily store these and we can transmit them so the the purpose of serialization and deserialization to convert objects to a format that can be easily stored or transmitted and then reconstruct them back into their original form and the use case of serialization is that saving and loading of trading configuration files logging trade activities and transmitting data from different components of a trading system and now let’s discuss the purpose and use case of pickling so with the help of pickling we can convert python objects to a bite stream or vice versa which is efficient for storing and transmitting binary data and when it comes to the use case we can use this and saving and loading of complex objects such as train models large data sets or trading strategies that needs to be preserved across sessions so enough of talking and now let’s dive into the coding examples so you can see here that serialization is the process of converting python data types to Json format and D serialization is the process of converting Json to python data types this is the simplest definition and you just need to know this nothing else means we will simply convert any data type it can be string dictionary list to a Jon format and again in the process of D calization we’ll convert back from that Json format to our data type right.
Watch this Day 16 video tutorial
Day 16: Serialization, Deserialization & Pickling
