(SOLVED) How do i pull this JSON?

Development Software Development
,
1

data:
https://www.halowaypoint.com/en-us/games/halo-the-master-chief-collection/xbox-one/game-history?gamertags=kenkoda&gameVariant=all&view=DataOnly

I want to pull this down so i can work with it in python, id like to parse the last 10 games of any player and display the K/D that they are currently playing at.

i found that the site has a view data area that presents the data but i cant find a way to just get the bits im looking for, i can curl it all down and work with it if i have to but i feel like im missing something.

id feel dumb if i just havent seen this thing before and it has a known API or something o_o

2

The data appears to be behind a login so you will need to use the api login token or set auth headers with what ever library you are using to fetch the data.

3

yeah, ill have to figure that out too. im more or less asking if anyone has seen this type of page before and if it has any way to request via its own api as the overall site has no api

4

oh, one sec

5

yeah once I signed in you got the raw json output so you can just use that as the data source.

6

Is this the api you were looking for?

https://developer.haloapi.com/

7

as far as i could find they only return data for Halo5, im trying to work with MCC.

why they dont have this as an open API is hard to understand

8

this needed a little work to bring in to working order but its working now.

#!/usr/bin/python3
__author__ = 'Damon Pollard (@DamonLPollard)'

import re
import random
import json
import requests

WLID_USERNAME = "[email protected]"
WLID_PASSWORD = "Mr.skibitybibityboppts"
USER_AGENT = 'Mozilla/5.0 (Windows NT 6.3; WOW64) ' \
             'AppleWebKit/537.36 (KHTML, like Gecko) ' \
             'Chrome/38.0.2125.104 Safari/537.36'

R_PPFT = "<input type=\"hidden\" name=\"PPFT\" id=\"i0327\" value=\"(.+?)\"\/>"
R_PPSX = "L:'(.+?)'"
R_URLPOST = "urlPost:'(.+?)'"

s = requests.Session()

oauth20_authorize = "https://login.live.com/oauth20_authorize.srf?client_id=000000004C0BD2F1&scope=xbox.basic+xbox.of" \
                    "fline_access&response_type=code&redirect_uri=https:%2f%2fwww.halowaypoint.com%2fauth%2fcallback&" \
                    "locale=en-us&display=touch&state=https%253a%252f%252fwww.halowaypoint.com%252fen-us%252fgames%25" \
                    "2fhalo-the-master-chief-collection%252fxbox-one%252fgame-history%253fgamertags%253dkenkoda%2" \
                    "526view%253dDataOnly"

response_one = s.get(oauth20_authorize,
                     headers={
                         'user-agent': USER_AGENT,
                         'host': 'login.live.com'
                     },
                     verify=True
    )

    ppft = re.search(R_PPFT, response_one.text).group(1)
    ppsx = re.search(R_PPSX, response_one.text)
    post = re.search(R_URLPOST, response_one.text).group(1)

    response_two = s.post(post,

                      data={
                          'PPFT': ppft,
                          'login': WLID_USERNAME,
                          'passwd': WLID_PASSWORD,
                          'LoginOptions': '3',
                          'NewUser': '1',
                          'PPSX': ppsx,
                          'type': '11',
                          'i3': random.randrange(5000, 10000),
                          'm1': '1920',
                          'm2': '1080',
                          'm3': '0',
                          'i12': '1',
                          'i17': '0',
                          'i18': '__MobileLogin|1,',
                      },
                      headers={
                          'user-agent': USER_AGENT,
                          'referer': oauth20_authorize,
                          'host': 'login.live.com',
                          'origin': 'https://login.live.com'
                      },
                      verify=True,
                      allow_redirects=False
)
callback_url = response_two.headers['Location']

response_three = s.get(callback_url,
                       headers={
                           'user-agent': USER_AGENT,
                           'referer': oauth20_authorize,
                           'host': 'www.halowaypoint.com'
                       },
                       verify=True,
                       allow_redirects=False
)
endpoint = response_three.headers['Location']

response_four = s.get(endpoint,
                      headers={
                          'user-agent': USER_AGENT,
                          'referer': oauth20_authorize,
                          'host': 'www.halowaypoint.com'
                      },
                      verify=True,
                      allow_redirects=False
)
data = response_four.json()
json = json.dumps(data, indent=4, sort_keys=True)
print(json)