Within the final put up we took a take a look at downloading songs from Soundcloud. On this put up we are going to check out Fb and the way we are able to create a downloader for Fb movies. It began with me desirous to obtain a video from Fb which I had the copyrights to. I needed to automate the method in order that I might obtain a number of movies with only one command.
Now there are instruments like
youtube-dl which might do that job for you however I needed to discover Fb’s API myself. With none additional ado let me present you step-by-step how I approached this mission. On this put up we are going to cowl downloading public movies. Within the subsequent put up I’ll check out downloading personal movies.
Step 1: Discovering a Video
Discover a video which you personal and have copyrights to. Now there are two varieties of movies on Fb. The primary sort is the general public movies which may be accessed by anybody after which there are personal movies that are accessible solely by a sure subset of individuals on Fb. Simply to maintain issues simple, I initially determined to make use of a public video with plans on increasing the system for personal movies afterwards.
Step 2: Recon
On this step we are going to open up the video in a brand new tab the place we aren’t logged in simply to see whether or not we are able to entry these public movies with out being logged in or not. I attempted doing it for the video in query and that is what I received:
Apparently we are able to’t entry the globally shared video as effectively with out logging in. Nonetheless, I remembered that I just lately noticed a video with out being logged in and that piqued my curiosity. I made a decision to discover the unique video a bit extra.
I right-clicked on the unique video simply to test it’s supply and to determine whether or not the video url was reconstructable utilizing the unique web page url. As a substitute of discovering the video supply, I discovered a distinct url which can be utilized to share this video. Check out these photos to get a greater understanding of what I’m speaking about:
I attempted opening this url in a brand new window with out being logged in and growth! The video opened! Now I’m not certain whether or not it labored simply by sheer luck or it truly is a legitimate method to view a video with out being logged in. I attempted this on a number of movies and it labored every single time. Both Means, we now have received a method to entry the video with out logging in and now it’s time to intercept the requests which Fb makes after we attempt to play the video.
Open up Chrome developer instruments and click on on the XHR button similar to this:
XHR stands for XMLHttpRequest and is utilized by the web sites to request extra knowledge utilizing Javascript as soon as the webpage has been loaded. Mozilla docs has a very good rationalization of it:
Use
XMLHttpRequest(XHR) objects to work together with servers. You may retrieve knowledge from a URL with out having to do a full web page refresh. This permits a Internet web page to replace simply a part of a web page with out disrupting what the person is doing.XMLHttpRequestis used closely in Ajax programming.
Filtering requests utilizing XHR permits us to chop down the variety of requests we must look by. It won’t work all the time so for those who don’t see something attention-grabbing after filtering out requests utilizing XHR, check out the “all” tab.
The XHR tab was attention-grabbing, it didn’t comprise any API request. As a substitute the very first requested hyperlink was the mp4 video itself.
This was surprizing as a result of often firms like Fb wish to have an intermediate server in order that they don’t must hardcore the mp4 hyperlinks within the webpage. Nonetheless, whether it is simple for me this manner then who am I to complain?
My very subsequent step was to seek for this url within the authentic supply of the web page and by chance I discovered it:
This confirmed my suspicions. Fb hardcores the video url within the authentic web page for those who view the web page with out signing in. We’ll late see how that is completely different when you find yourself signed in. The url in present case is present in a <script> tag.
Step 3: Automating it
Now let’s write a Python script to obtain public movies. The script is fairly easy. Right here is the code:
import requests as r
import re
import sys
url = sys.argv[-1]
html = r.get(url)
video_url = re.search('hd_src:"(.+?)"', html.textual content).group(1)
print(video_url)
Save the above code in a _videoobtain.py file and use it like this:
$ python video_download.py video_url
Don’t neglect to exchange video_url with precise video url of this manner:
https://www.fb.com/username/movies/10213942282701232/
The script will get the video url from the command line. It then opens up the video web page utilizing requests after which makes use of common expressions to parse the video url from the web page. This won’t work if the video isn’t accessible in HD. I go away that as much as you to determine how you can deal with that case.
That’s all for at the moment. I’ll cowl the downloading of your personal movies within the subsequent put up. That is a little more concerned and requires you logging into Fb. Comply with the weblog and keep tuned! When you have any questions/feedback/ideas please use the remark kind or e-mail me.
Have an excellent day!
