Table of Contents
Overview on Hidden Endpoints
Hidden endpoints are basically a treasure trove for all the Web Pentesters as most of the hidden endpoints are not in use or deprecated, so most of these endpoints are very vulnerable to different types of vulnerability attacks. Many organisations have faced critical vulnerability attacks through Hidden endpoints.
With that in mind, let's jump in to see how we can find the Hidden Endpoints.
There are various methods to find Hidden endpoints:
- Google Dorks
- Github
- Archive’s
- Apk’s
- JS Files
1. Google Dorks
Google has always been and will always be the major information gatherer. Google Dork isn’t as simple as a normal Google search. It uses advanced operators to find specific information such as versions, filenames, id’s in the search results. Any one with an Internet can easily learn about the various advanced operators on many public sources and use them to find many vulnerabilities in the existing systems.
Search Engines like Google and Bing support various operators to make search Queries which can be very useful for hackers to find many sensitive endpoints.
Some of the Google dorks we can use to find Hidden endpoints are below:
site:example.com ext:doc | ext:docx | ext:odt | ext:rtf | ext:sxw | ext:psw | ext:ppt | ext:pptx | ext:pps | ext:csv site:s3.amazonaws.com + example.com site:accounts.example.com inurl:user_id site:example.com inurl:login | inurl:signin | intitle:Login | intitle:"sign in"
2. Github
Github is a great tool which could be used to gather a lot of information about a company’s infrastructure. We can start off by just using the target’s name and we will be able to find lots of commits made in their repositories. We can find out if they use any Internal tools like JIRA , Asana or any other tools from their commits very easily.
Github recon is a very important phase of a Web application Pentest because Github repo’s store lots of private API keys and credentials needed for an application to run. Github repo’s also stores subdomains ,endpoints of API’s in their code and we can use Github dorks to find this sensitive information. Though I would say, it's always better to manually use Github dorks so that we can find much more data we can still use lot of publicly available Github recon tools to automate this process.
A few things may sometimes go wrong in this process of finding hidden endpoints:
- We may end up going out of the scope by discovering a third party app
- We may find very old keys or keys not used by the target
- We may find the endpoints found already removed or no longer in use.
Some of the Github dorks we can use to find Hidden endpoints are,
“example.tld” API_link “example.tld” API_key “example.tld” secret_key “example.tld” aws_key “example.tld” Password “example.tld” FTP “example.tld” login “example.tld” github_token
You can find more Github dorks on the internet very easily, you just need to play around with the dorks and find your target organisation repo and use the dorks to filter the information to find the hidden endpoints and sensitive secrets.
3. APK’s
Android Application Package File (APK) is the file format used to install application software onto the Android operating system. Apk’s can be decompiled and we can go through all the code files to find hidden endpoints. Though this can be done manually, it's better to use automatic tools for faster and efficient process. One such tool which can be used to find hidden endpoints from Apk’s is ApkLeaks.
3.1 ApkLeaks
Apkleaks is an open source python based tool which can be used to find various hidden endpoints , secrets in an Apk application. It’s very fast , efficient and is easily available on various Linux distributions.
3.1.1 Installing ApkLeaks
You can download Apkleaks in 2 ways:
- pip3 package manager
- Source
Install using pip3
pip3 install apkleaks
Install using source
git clone https://github.com/dwisiswant0/apkleaks cd apkleaks/ pip3 install -r requirements.txt
3.1.2 Using ApkLeaks to find endpoints
Scanning an APK file to find endpoints
python3 apkleaks.py -f file.apk
Writing the results into a file
python3 apkleaks.py -f /home/spi3er/Downloads/Templerun.apk -o ../output.txt _ ____ _ ___ _ / \ | _ \| |/ / | ___ __ _| | _____ / _ \ | |_) | ' /| | / _ \/ _` | |/ / __| / ___ \| __/| . \| |__| __/ (_| | <\__ \ /_/ \_\_| |_|\_\_____\___|\__,_|_|\_\___/ v2.6.1 -- Scanning APK file for URIs, endpoints & secrets (c) 2020-2021, dwisiswant0 ** Decompiling APK... INFO - loading ... INFO - processing ... INFO - done ** Scanning against 'com.disney.TempleRunOz.goo' [Artifactory_Password] - AP2tMkmTC0clySJvgUxUmczyyQU [Facebook_Secret_Key] - FB_APP_SIGNATURE = "30820268308201d102044a9c4610300d [IP_Address] - 1.25.0.3 - 10.0.1.7 - 10.0.2.2 - 192.168.1.1 - 192.168.1.8 - 192.168.2.1 [LinkFinder] - /1.1/statuses/update_with_media.json - /Android/data/ - /analytics - /analytics/ - /android_v2/handle_app_loads - /android_v2/handle_crashes - /android_v2/handle_exceptions - /android_v2/ndk_crash - /android_v2/update_package_name - /android_v2/update_user_metadata - /cache - /com.crittercism/lib/ - /files - /foo/bar/dumdum - /foobar/workspace/ - /forum/springboard - /friends - /libcrittercism-ndk.so - /me/ - /proc/cpuinfo - /proc/meminfo - /sdcard - /sdcard/NSFileManagerTests - /sdcard/NSFileManagerTestsDestination - /sdcard/NSFileManagerTestsSource - /strings/ - /tmp - AES/CBC/PKCS5Padding - AES/ECB/NoPadding - Android/data - OZ/Tinted_Alpha_Font - Oz/Materials/oz_ww_master_opaque - Prefabs/Temple/environments/darkforest/oz_df_master_opaque - Prefabs/Temple/environments/emeraldcity/oz_ec_master_opaque - assets/bin/ - bin/Data/settings.xml - challenges/team - config.json - content/unknown - content://com.facebook.katana.provider.AttributionIdProvider - curly.txt - http://api.kaixin001.com/oauth/access_token - http://api.kaixin001.com/oauth/authorize?oauth_token=%s - http://api.kaixin001.com/oauth/request_token - http://api.t.163.com/oauth/access_token - http://api.t.163.com/oauth/authenticate?oauth_token=%s - http://api.t.163.com/oauth/authorize?oauth_token=%s - http://api.t.163.com/oauth/request_token - http://api.t.sina.com.cn/oauth/access_token - http://api.t.sina.com.cn/oauth/authorize?oauth_token=%s - http://api.t.sina.com.cn/oauth/request_token - http://api.t.sohu.com/oauth/access_token - http://api.t.sohu.com/oauth/authorize?oauth_token=%s - http://api.t.sohu.com/oauth/request_token - http://api.twitter.com - http://api.twitter.com/oauth/access_token - http://api.twitter.com/oauth/request_token - http://disneynetwork0-a.akamaihd.net/mobilenetwork/referralstore/bootstrap/ - http://foursquare.com/oauth/access_token - http://foursquare.com/oauth/authorize?oauth_token=%s - http://foursquare.com/oauth/request_token - http://java.sun.com/j2se/1.3/ - http://openapi.lovefilm.com/oauth/access_token - http://openapi.lovefilm.com/oauth/request_token - http://schemas.android.com/apk/res/android - http://vimeo.com/oauth/access_token - http://vimeo.com/oauth/authorize?oauth_token=%s - http://vimeo.com/oauth/request_token - http://www.amazon.com/gp/mas/get-appstore/android/ref=mas_mx_mba_iap_dl - http://www.apple.com/DTDs/PropertyList-1.0.dtd - http://www.apple.com/DTDs/PropertyList-1.0.dtd\ - http://www.burstly.com/scheme - http://www.foo.com - http://www.plurk.com/OAuth/access_token - http://www.plurk.com/OAuth/authorize?oauth_token=%s - http://www.plurk.com/OAuth/request_token - http://www.plurk.com/m/authorize?oauth_token=%s - http://www.springframework.org/schema/beans - http://www.springframework.org/schema/p - http://www.springframework.org/schema/security - http://www.springframework.org/schema/util - http://www.texturepacker.com - http://www.w3.org/2001/XMLSchema-instance - https://.facebook.com - https://api.crittercism.com - https://api.disney.com/dismo/bi/v1 - https://api.disney.com/mobilenetwork/referralstore/v1/config - https://api.dropbox.com/0/oauth/access_token - https://api.dropbox.com/0/oauth/request_token - https://api.facebook.com/method/ - https://api.facebook.com/restserver.php - https://api.linkedin.com/uas/oauth/accessToken - https://api.linkedin.com/uas/oauth/authorize?oauth_token=%s - https://api.linkedin.com/uas/oauth/requestToken - https://api.login.yahoo.com/oauth/v2/get_request_token - https://api.login.yahoo.com/oauth/v2/get_token - https://api.login.yahoo.com/oauth/v2/request_auth?oauth_token=%s - https://api.twitter.com - https://api.twitter.com/1.1/account/verify_credentials.json - https://api.twitter.com/oauth/access_token - https://api.twitter.com/oauth/authenticate?oauth_token=%s - https://api.twitter.com/oauth/authorize?oauth_token=%s - https://api.twitter.com/oauth/request_token - https://api.vkontakte.ru/oauth/access_token - https://api.vkontakte.ru/oauth/authorize?client_id=%s&redirect_uri=%s&response_type=code - https://facebook.com - https://foursquare.com/oauth2/access_token?grant_type=authorization_code - https://foursquare.com/oauth2/authenticate?client_id=%s&response_type=code&redirect_uri=%s - https://graph.facebook.com - https://graph.facebook.com/ - https://graph.facebook.com/%s/picture - https://graph.facebook.com/oauth/access_token - https://id.sapo.pt/oauth/access_token - https://id.sapo.pt/oauth/authorize?oauth_token=%s - https://id.sapo.pt/oauth/request_token - https://m.facebook.com/dialog/ - https://oauth.constantcontact.com/ws/oauth/access_token - https://oauth.constantcontact.com/ws/oauth/confirm_access?oauth_token=%s - https://oauth.constantcontact.com/ws/oauth/request_token - https://oauth.live.com/authorize?client_id=%s&redirect_uri=%s&response_type=code - https://oauth.live.com/authorize?client_id=%s&redirect_uri=%s&response_type=code&scope=%s - https://oauth.live.com/token?grant_type=authorization_code - https://open.t.qq.com/cgi-bin/access_token - https://open.t.qq.com/cgi-bin/authorize?oauth_token=%s - https://open.t.qq.com/cgi-bin/request_token - https://sandbox.evernote.com/oauth - https://sandbox.evernote.com/oauth?oauth_token=%s - https://www.appcred.com - https://www.dropbox.com/0/oauth/authorize?oauth_token= - https://www.evernote.com/OAuth.action?oauth_token=%s - https://www.evernote.com/oauth - https://www.facebook.com/dialog/oauth?client_id=%s&redirect_uri=%s - https://www.facebook.com/dialog/oauth?client_id=%s&redirect_uri=%s&scope=%s - https://www.facebook.com/impression.php - https://www.google.com/accounts/OAuthAuthorizeToken?oauth_token=%s - https://www.google.com/accounts/OAuthGetAccessToken - https://www.google.com/accounts/OAuthGetRequestToken - https://www.lovefilm.com/activate?oauth_token=%s - https://www.yammer.com/oauth/access_token - https://www.yammer.com/oauth/request_token - larry.txt - latestSummary.txt - me/feed - me/friends - me/permissions - me/photos - me/videos - moe.txt - pendingUploadDescription.txt - https://www.yammer.com/oauth/authorize?oauth_token=%s ** Results saved into '../output.txt'.
Saving the results file in JSON format
apkleaks -f file.apk -o results.json --json
4. Archives
Archive websites store a lot of information about all the websites present and previous data. They contain petabytes of data which can be used by anyone who has an Internet connection.Some of the most useful archives are AlienVault,Common Crawl and WaybackMachine.
Web Pentesters can use these Archive’s data for their own use in finding the hidden endpoints. The most popular tools used by Web Pentesters for finding the hidden endpoints through Archive data are Gau and WaybackUrls.
4.1 Gau
GetallUrls (gau) is an open source tool written in Go which fetches all urls from AlienVault , CommonCrawl, Waybackmachine for any given domain. It’s fast and can be easily piped with many other pentesting tools to find many hidden endpoints.
4.1.1 Installing Gau
Check if Go is installed or not
Go version
Install Gau by using this command
go install github.com/lc/gau/v2/cmd/gau@latest
Check if Gau is working fine or not
gau -version
4.1.2 Using Gau to find hidden endpoints
Supply a text file with domains to search for
cat domains.txt | gau > endpoints.txt
Writing the results into a file
gau --o out.txt
Blacklisting png, jpg, gif from the scanning
gau --blacklist jpg,gif,png domain.com
5. JS Files
JS files are a prime source of hidden endpoints because every website on the internet uses JS for loading their files whether it be some third party applications or interacting between any webpages. These JS files can be statically analysed for a lot of endpoints but doing this manually will take forever, thankfully we have got some very nice tools which will automate this process.
The best tools I personally use for finding endpoints from JSFiles are LinkFinder, JSFScan. So let’s get our hands dirty by using these amazing tools.
5.1 LinkFinder
Linkfinder is an open source tool written in python for finding parameters and endpoints in Javascript files. Linkfinder does so by using jsbeautifier for python with a large regular expression. The regular expression is responsible for finding,
- Full URLs
- Absolute URLs
- Relative URLs with at least one slash
- Relative URLs without a slash
5.1.1 Installing LinkFinder
Run the following commands to install LinkFinder
git clone https://github.com/GerbenJavado/LinkFinder.git cd LinkFinder python3 setup.py install
5.1.2 Using LinkFinder to find hidden endpoints
Finding endpoints from a URL
python3 linkfinder.py -i https://example.com/index.js -o cli
Displaying the output as html
python3 linkfinder.py -i https://example.com/index.js -o results.html
Analyse enter domain for Js Files
python3 linkfinder.py -d todoist.com
5.2 JSFScan
JSFscan can be called an ultimate tool when you are looking for Javascript recon i.e you are able to use JSFscan to find endpoints, secrets, variables, create wordlists from the Js files and many more.
Pre-requisities:
- Gau
- Httpx
You have already seen how to install Gau in the previous part, so do it accordingly.
5.2.1 JSFScan Installation
Install Httpx using the following command,
go install -v github.com/projectdiscovery/httpx/cmd/httpx@latest
Install JSFScan using the following commands,
git clone https://github.com/KathanP19/JSFScan.sh.git cd JSFScan.sh sudo chmod +x install.sh ./install.sh
5.2.2 Using JSFScan to find hidden endpoints
Import file containing JS Urls
Bash JSFScan.sh -f ./path/to/JSUrls
Find endpoints from JSUrls
Bash JSFScan.sh -e
Find secrets from JsUrls
Bash JSFScan.sh -s
Summary
In this post , we have learnt about 5 ways to find hidden endpoints during a Web application Pentest. Hidden endpoints are not just found only by these 5 ways , they can still be found through Trello Boards and some other ways but I have listed the most important and efficient ones in this article.If you are just getting started in security , then please check out our articles of Ethical Hacking on our website.If you encounter issues in any of the commands above, please let us know in the comments below.