100% proven ways to find hidden endpoints [Tutorial]

Overview on Hidden Endpoints

Hidden endpoints are basically a treasure trove for all the Web Pentesters as most of the hidden endpoints are not in use or deprecated, so most of these endpoints are very vulnerable to different types of vulnerability attacks. Many organisations have faced critical vulnerability attacks through Hidden endpoints.

With that in mind, let's jump in to see how we can find the Hidden Endpoints.

There are various methods to find Hidden endpoints:

1. Google Dorks
2. Github
3. Archive’s
4. Apk’s
5. JS Files

1. Google Dorks

Google has always been and will always be the major information gatherer. Google Dork isn’t as simple as a normal Google search. It uses advanced operators to find specific information such as versions, filenames, id’s in the search results. Any one with an Internet can easily learn about the various advanced operators on many public sources and use them to find many vulnerabilities in the existing systems.

Search Engines like Google and Bing support various operators to make search Queries which can be very useful for hackers to find many sensitive endpoints.

Some of the Google dorks we can use to find Hidden endpoints are below:

site:example.com ext:doc | ext:docx | ext:odt | ext:rtf | ext:sxw | ext:psw | ext:ppt | ext:pptx | ext:pps | ext:csv 
site:s3.amazonaws.com + example.com
site:accounts.example.com inurl:user_id
site:example.com inurl:login | inurl:signin | intitle:Login | intitle:"sign in"

2. Github

Github is a great tool which could be used to gather a lot of information about a company’s infrastructure. We can start off by just using the target’s name and we will be able to find lots of commits made in their repositories. We can find out if they use any Internal tools like JIRA , Asana or any other tools from their commits very easily.

Github recon is a very important phase of a Web application Pentest because Github repo’s store lots of private API keys and credentials needed for an application to run. Github repo’s also stores subdomains ,endpoints of API’s in their code and we can use Github dorks to find this sensitive information. Though I would say, it's always better to manually use Github dorks so that we can find much more data we can still use lot of publicly available Github recon tools to automate this process.

A few things may sometimes go wrong in this process of finding hidden endpoints:

• We may end up going out of the scope by discovering a third party app
• We may find very old keys or keys not used by the target
• We may find the endpoints found already removed or no longer in use.

Some of the Github dorks we can use to find Hidden endpoints are,

“example.tld” API_link 
“example.tld” API_key
“example.tld” secret_key
“example.tld” aws_key
“example.tld” Password 
“example.tld” FTP
“example.tld” login 
“example.tld” github_token 

You can find more Github dorks on the internet very easily, you just need to play around with the dorks and find your target organisation repo and use the dorks to filter the information to find the hidden endpoints and sensitive secrets.

3. APK’s

Android Application Package File (APK) is the file format used to install application software onto the Android operating system. Apk’s can be decompiled and we can go through all the code files to find hidden endpoints. Though this can be done manually, it's better to use automatic tools for faster and efficient process. One such tool which can be used to find hidden endpoints from Apk’s is ApkLeaks.

3.1 ApkLeaks

Apkleaks is an open source python based tool which can be used to find various hidden endpoints , secrets in an Apk application. It’s very fast , efficient and is easily available on various Linux distributions.

3.1.1 Installing ApkLeaks

You can download Apkleaks in 2 ways:

1. pip3 package manager
2. Source

Install using pip3

pip3 install apkleaks

Install using source

git clone https://github.com/dwisiswant0/apkleaks

cd apkleaks/

pip3 install -r requirements.txt

3.1.2 Using ApkLeaks to find endpoints

Scanning an APK file to find endpoints

python3 apkleaks.py -f file.apk

Writing the results into a file

python3 apkleaks.py -f /home/spi3er/Downloads/Templerun.apk -o ../output.txt
_ ____ _ ___ _
/ \ | _ \| |/ / | ___ __ _| | _____
/ _ \ | |_) | ' /| | / _ \/ _` | |/ / __|
/ ___ \| __/| . \| |__| __/ (_| | <\__ \
/_/ \_\_| |_|\_\_____\___|\__,_|_|\_\___/
v2.6.1
--
Scanning APK file for URIs, endpoints & secrets
(c) 2020-2021, dwisiswant0

** Decompiling APK...
INFO - loading ...
INFO - processing ...
INFO - done

** Scanning against 'com.disney.TempleRunOz.goo'

[Artifactory_Password]
- AP2tMkmTC0clySJvgUxUmczyyQU

[Facebook_Secret_Key]
- FB_APP_SIGNATURE = "30820268308201d102044a9c4610300d

[IP_Address]
- 1.25.0.3
- 10.0.1.7
- 10.0.2.2
- 192.168.1.1
- 192.168.1.8
- 192.168.2.1

[LinkFinder]
- /1.1/statuses/update_with_media.json
- /Android/data/
- /analytics
- /analytics/
- /android_v2/handle_app_loads
- /android_v2/handle_crashes
- /android_v2/handle_exceptions
- /android_v2/ndk_crash
- /android_v2/update_package_name
- /android_v2/update_user_metadata
- /cache
- /com.crittercism/lib/
- /files
- /foo/bar/dumdum
- /foobar/workspace/
- /forum/springboard
- /friends
- /libcrittercism-ndk.so
- /me/
- /proc/cpuinfo
- /proc/meminfo
- /sdcard
- /sdcard/NSFileManagerTests
- /sdcard/NSFileManagerTestsDestination
- /sdcard/NSFileManagerTestsSource
- /strings/
- /tmp
- AES/CBC/PKCS5Padding
- AES/ECB/NoPadding
- Android/data
- OZ/Tinted_Alpha_Font
- Oz/Materials/oz_ww_master_opaque
- Prefabs/Temple/environments/darkforest/oz_df_master_opaque
- Prefabs/Temple/environments/emeraldcity/oz_ec_master_opaque
- assets/bin/
- bin/Data/settings.xml
- challenges/team
- config.json
- content/unknown
- content://com.facebook.katana.provider.AttributionIdProvider
- curly.txt
- http://api.kaixin001.com/oauth/access_token
- http://api.kaixin001.com/oauth/authorize?oauth_token=%s
- http://api.kaixin001.com/oauth/request_token
- http://api.t.163.com/oauth/access_token
- http://api.t.163.com/oauth/authenticate?oauth_token=%s
- http://api.t.163.com/oauth/authorize?oauth_token=%s
- http://api.t.163.com/oauth/request_token
- http://api.t.sina.com.cn/oauth/access_token
- http://api.t.sina.com.cn/oauth/authorize?oauth_token=%s
- http://api.t.sina.com.cn/oauth/request_token
- http://api.t.sohu.com/oauth/access_token
- http://api.t.sohu.com/oauth/authorize?oauth_token=%s
- http://api.t.sohu.com/oauth/request_token
- http://api.twitter.com
- http://api.twitter.com/oauth/access_token
- http://api.twitter.com/oauth/request_token
- http://disneynetwork0-a.akamaihd.net/mobilenetwork/referralstore/bootstrap/
- http://foursquare.com/oauth/access_token
- http://foursquare.com/oauth/authorize?oauth_token=%s
- http://foursquare.com/oauth/request_token
- http://java.sun.com/j2se/1.3/
- http://openapi.lovefilm.com/oauth/access_token
- http://openapi.lovefilm.com/oauth/request_token
- http://schemas.android.com/apk/res/android
- http://vimeo.com/oauth/access_token
- http://vimeo.com/oauth/authorize?oauth_token=%s
- http://vimeo.com/oauth/request_token
- http://www.amazon.com/gp/mas/get-appstore/android/ref=mas_mx_mba_iap_dl
- http://www.apple.com/DTDs/PropertyList-1.0.dtd
- http://www.apple.com/DTDs/PropertyList-1.0.dtd\
- http://www.burstly.com/scheme
- http://www.foo.com
- http://www.plurk.com/OAuth/access_token
- http://www.plurk.com/OAuth/authorize?oauth_token=%s
- http://www.plurk.com/OAuth/request_token
- http://www.plurk.com/m/authorize?oauth_token=%s
- http://www.springframework.org/schema/beans
- http://www.springframework.org/schema/p
- http://www.springframework.org/schema/security
- http://www.springframework.org/schema/util
- http://www.texturepacker.com
- http://www.w3.org/2001/XMLSchema-instance
- https://.facebook.com
- https://api.crittercism.com
- https://api.disney.com/dismo/bi/v1
- https://api.disney.com/mobilenetwork/referralstore/v1/config
- https://api.dropbox.com/0/oauth/access_token
- https://api.dropbox.com/0/oauth/request_token
- https://api.facebook.com/method/
- https://api.facebook.com/restserver.php
- https://api.linkedin.com/uas/oauth/accessToken
- https://api.linkedin.com/uas/oauth/authorize?oauth_token=%s
- https://api.linkedin.com/uas/oauth/requestToken
- https://api.login.yahoo.com/oauth/v2/get_request_token
- https://api.login.yahoo.com/oauth/v2/get_token
- https://api.login.yahoo.com/oauth/v2/request_auth?oauth_token=%s
- https://api.twitter.com
- https://api.twitter.com/1.1/account/verify_credentials.json
- https://api.twitter.com/oauth/access_token
- https://api.twitter.com/oauth/authenticate?oauth_token=%s
- https://api.twitter.com/oauth/authorize?oauth_token=%s
- https://api.twitter.com/oauth/request_token
- https://api.vkontakte.ru/oauth/access_token
- https://api.vkontakte.ru/oauth/authorize?client_id=%s&redirect_uri=%s&response_type=code
- https://facebook.com
- https://foursquare.com/oauth2/access_token?grant_type=authorization_code
- https://foursquare.com/oauth2/authenticate?client_id=%s&response_type=code&redirect_uri=%s
- https://graph.facebook.com
- https://graph.facebook.com/
- https://graph.facebook.com/%s/picture
- https://graph.facebook.com/oauth/access_token
- https://id.sapo.pt/oauth/access_token
- https://id.sapo.pt/oauth/authorize?oauth_token=%s
- https://id.sapo.pt/oauth/request_token
- https://m.facebook.com/dialog/
- https://oauth.constantcontact.com/ws/oauth/access_token
- https://oauth.constantcontact.com/ws/oauth/confirm_access?oauth_token=%s
- https://oauth.constantcontact.com/ws/oauth/request_token
- https://oauth.live.com/authorize?client_id=%s&redirect_uri=%s&response_type=code
- https://oauth.live.com/authorize?client_id=%s&redirect_uri=%s&response_type=code&scope=%s
- https://oauth.live.com/token?grant_type=authorization_code
- https://open.t.qq.com/cgi-bin/access_token
- https://open.t.qq.com/cgi-bin/authorize?oauth_token=%s
- https://open.t.qq.com/cgi-bin/request_token
- https://sandbox.evernote.com/oauth
- https://sandbox.evernote.com/oauth?oauth_token=%s
- https://www.appcred.com
- https://www.dropbox.com/0/oauth/authorize?oauth_token=
- https://www.evernote.com/OAuth.action?oauth_token=%s
- https://www.evernote.com/oauth
- https://www.facebook.com/dialog/oauth?client_id=%s&redirect_uri=%s
- https://www.facebook.com/dialog/oauth?client_id=%s&redirect_uri=%s&scope=%s
- https://www.facebook.com/impression.php
- https://www.google.com/accounts/OAuthAuthorizeToken?oauth_token=%s
- https://www.google.com/accounts/OAuthGetAccessToken
- https://www.google.com/accounts/OAuthGetRequestToken
- https://www.lovefilm.com/activate?oauth_token=%s
- https://www.yammer.com/oauth/access_token
- https://www.yammer.com/oauth/request_token
- larry.txt
- latestSummary.txt
- me/feed
- me/friends
- me/permissions
- me/photos
- me/videos
- moe.txt
- pendingUploadDescription.txt
- https://www.yammer.com/oauth/authorize?oauth_token=%s

** Results saved into '../output.txt'.

Saving the results file in JSON format

apkleaks -f file.apk -o results.json --json

4. Archives

Archive websites store a lot of information about all the websites present and previous data. They contain petabytes of data which can be used by anyone who has an Internet connection.Some of the most useful archives are AlienVault,Common Crawl and WaybackMachine.

Web Pentesters can use these Archive’s data for their own use in finding the hidden endpoints. The most popular tools used by Web Pentesters for finding the hidden endpoints through Archive data are Gau and WaybackUrls.

4.1 Gau

GetallUrls (gau) is an open source tool written in Go which fetches all urls from AlienVault , CommonCrawl, Waybackmachine for any given domain. It’s fast and can be easily piped with many other pentesting tools to find many hidden endpoints.

4.1.1 Installing Gau

Check if Go is installed or not

  Go version

Install Gau by using this command

go install github.com/lc/gau/v2/cmd/gau@latest

Check if Gau is working fine or not

gau -version

4.1.2 Using Gau to find hidden endpoints

Supply a text file with domains to search for

cat domains.txt | gau > endpoints.txt

Writing the results into a file

gau --o out.txt

Blacklisting png, jpg, gif from the scanning

gau --blacklist jpg,gif,png domain.com

5. JS Files

JS files are a prime source of hidden endpoints because every website on the internet uses JS for loading their files whether it be some third party applications or interacting between any webpages. These JS files can be statically analysed for a lot of endpoints but doing this manually will take forever, thankfully we have got some very nice tools which will automate this process.

The best tools I personally use for finding endpoints from JSFiles are LinkFinder, JSFScan. So let’s get our hands dirty by using these amazing tools.

5.1 LinkFinder

Linkfinder is an open source tool written in python for finding parameters and endpoints in Javascript files. Linkfinder does so by using jsbeautifier for python with a large regular expression. The regular expression is responsible for finding,

• Full URLs
• Absolute URLs
• Relative URLs with at least one slash
• Relative URLs without a slash

5.1.1 Installing LinkFinder

Run the following commands to install LinkFinder

 git clone https://github.com/GerbenJavado/LinkFinder.git

 cd LinkFinder

 python3 setup.py install

5.1.2 Using LinkFinder to find hidden endpoints

Finding endpoints from a URL

python3 linkfinder.py -i https://example.com/index.js -o cli

Displaying the output as html

python3 linkfinder.py -i https://example.com/index.js -o results.html

Analyse enter domain for Js Files

python3 linkfinder.py -d todoist.com

5.2 JSFScan

JSFscan can be called an ultimate tool when you are looking for Javascript recon i.e you are able to use JSFscan to find endpoints, secrets, variables, create wordlists from the Js files and many more.

Pre-requisities:

1. Gau
2. Httpx

You have already seen how to install Gau in the previous part, so do it accordingly.

5.2.1 JSFScan Installation

Install Httpx using the following command,

go install -v github.com/projectdiscovery/httpx/cmd/httpx@latest

Install JSFScan using the following commands,

git clone https://github.com/KathanP19/JSFScan.sh.git

cd JSFScan.sh
 
sudo chmod +x install.sh

 ./install.sh

5.2.2 Using JSFScan to find hidden endpoints

Import file containing JS Urls

Bash JSFScan.sh -f ./path/to/JSUrls

Find endpoints from JSUrls

Bash JSFScan.sh -e

Find secrets from JsUrls

Bash JSFScan.sh -s

Summary

In this post , we have learnt about 5 ways to find hidden endpoints during a Web application Pentest. Hidden endpoints are not just found only by these 5 ways , they can still be found through Trello Boards and some other ways but I have listed the most important and efficient ones in this article.If you are just getting started in security , then please check out our articles of Ethical Hacking on our website.If you encounter issues in any of the commands above, please let us know in the comments below.