Unable to get the Team ID and Player ID using FBref() function

[PGupta-Git]

As the title says, I am unable to retrieve the Team ID and Player ID columns after scraping data with FBRef, which is making deduplication difficult as I am unable to merge the dataframe with a UUID, resulting in many duplicate values. Hence, want to know if this is a bug or justan implementation issue from my end

scrape_stats(year: str, league: str, stat_category: str) → dict[str, DataFrame]

Scrapes a single stats category

Adds team and player ID columns to the stats tables

Parameters:

        year (str) – See the [FBref](https://scraperfc.readthedocs.io/en/latest/year_parameter.html#fbref-year) year parameter docs for details.

        league (str) – See the src/ScraperFC/comps.yaml file for all available competitions.

        stat_category (str) – The stat category to scrape.

Returns:

    (squad_stats, opponent_stats, player_stats). Tuple elements will be None if the squad stats category does not contain data for the given year and league.
Return type:

    tuple[pd.DataFrame | None]

[oseymour]

Hey @PGupta-Git ! I'm confused, are you saying the columns aren't in the dataframes being returned, or that they don't have any values?

Can you please share the dataframes that you're getting? And what columns they have?

[PGupta-Git]

england premier league 2024 vs team data.json
england premier league 2024 team data.json
england premier league 2024 player data.json

I am trying to work off from the example provided at https://github.com/jakeyk11/football-data-analytics/blob/main/projects/00_data_import_and_misc_work/import_data_fbref.py and have adapted the script to adapt to the latest scraperfc API but when the scripts comes to the point where it checks for Team ID and Player ID, the script fails because these columns are no longer being returned. I tried joining with other keys, the scripts caches the data but I end up with some duplicated data due to joining them on Squad

Iterate over statistic type

for idx, statistic_group in enumerate(list(fbref_dict.keys())):

# Team stats for
temp_team_stat_for_df = fbref_dict[statistic_group][0].copy()
new_col_names = []
for col_name in temp_team_stat_for_df.columns:
    col_name_1 = '' if 'Unnamed' in col_name[0] else col_name[0]
    col_name_2 = col_name[1] if 'Unnamed' in col_name[0] else ' ' + col_name[1]
    new_col_names.append((col_name_1 + col_name_2).strip())
temp_team_stat_for_df.columns = new_col_names
if idx != 0:
    teaminfo_for_df = teaminfo_for_df.merge(temp_team_stat_for_df, left_on='Team ID', right_on='Team ID', suffixes=('', '_duplicate'), how = "outer") 
else:
    teaminfo_for_df = temp_team_stat_for_df

# Team stats against
temp_team_stat_against_df = fbref_dict[statistic_group][1].copy()
new_col_names = []
for col_name in temp_team_stat_against_df.columns:
    col_name_1 = '' if 'Unnamed' in col_name[0] else col_name[0]
    col_name_2 = col_name[1] if 'Unnamed' in col_name[0] else ' ' + col_name[1]
    new_col_names.append((col_name_1 + col_name_2).strip())
temp_team_stat_against_df.columns = new_col_names
if idx != 0:
    teaminfo_against_df = teaminfo_against_df.merge(temp_team_stat_against_df, left_on='Team ID', right_on='Team ID', suffixes=('', '_duplicate'), how = "outer") 
else:
    teaminfo_against_df = temp_team_stat_against_df

# Player stats
temp_player_stat_df = fbref_dict[statistic_group][2].copy()
new_col_names = []
for col_name in temp_player_stat_df.columns:
    col_name_1 = '' if 'Unnamed' in col_name[0] else col_name[0]
    col_name_2 = col_name[1] if 'Unnamed' in col_name[0] else ' ' + col_name[1]
    new_col_names.append((col_name_1 + col_name_2).strip())
temp_player_stat_df.columns = new_col_names
if idx != 0:
    playerinfo_df = playerinfo_df.merge(temp_player_stat_df, left_on=['Player', 'Player ID', 'Squad'], right_on=['Player', 'Player ID', 'Squad'], suffixes=('', '_duplicate'), how = "outer") 
else:
    playerinfo_df = temp_player_stat_df

[oseymour]

Got it, that makes sense. I'll put adding those player and team ID columns to these dataframes on the list of features to add! I'll let you know here when I've got something released.

[PGupta-Git]

I would be highly grateful. Thanks a lot.

[oseymour]

Hey @PGupta-Git I just released v4.1.0 which adds player and team ID columns to the dataframes in scrspe_match() and scrape_stats(). Can you let me know if this meets your needs?

[PGupta-Git]

Hi,

Yes, it is working well now. Thank you for such a quick implementation.

[oseymour]

Glad it's working! Feel free to reopen this if you need any other functionality or find any bugs