Table Of Contents

Dataframe Subset


The Subset utility allows you to easily generate subsets of pandas dataframes.



You must import a dfc dataframe to proceed with the subset utility.


Dataframe Subsets


A Subset is a collection of rows and columns in a dataframe selected via booean indexing.

A Subset is generated via defining a set of columns and rows selected from an input df.

A Subset can be continously generated by using the output df of the current subset as the input df fro the next subset.

After each subset is generated you can inspect the resultant dataframe to help determine what to define for the next subset.

Inspect Subset Dataframe


View Subset Dataframe In Excel



Defining, creating or runnning a dataframe subset consists of a simple 2 part process to generate a subset df.
The parts are deined below:


  • Subset Define Columns

    • Select the columns of the input dataframe to define the columns of the resultant subset df.

  • Subset Define Rows

    • Define the subset df criteria to define the rows of the resultant subset df.


Create Dataframe Subset

Define Subset Columns


Parameters:
*input_dataframe

The dfc dataframe to generate a subset from.

*column_names_list

The list of columns to keep or drop from the input_dataframe.

input_dataframe_column_names

The list of columns to select from to keep or drop.

action_for_columns_in_column_names_list

Boolean flag to keep or drop columns in column_names_list.

Commands: 
Define Subset Criteria

Go ahead and select the subset subset_selection_criteria.

Return

Exit the subset definition.

Help

Subset Help.

Columns Keep Drop




The above input examples for Keep and Drop will result in the same column list in the subset df.

Define Subset Rows

Once you select the columns of the subset df you need to define the subset title and the selection criteria.

The disabled sections of the input form represent code segmants that are automatically run by the dfcleanser. These segments provide the mechanism to define the dfc dataframes and make them visible to the dfcleanser. As you develop your criteria use the 'current_df' object to specify the input dfc dataframe.



Parameters:
*subset_dataframe_title

The dfc dataframe title of the subset dataframe generated.

*subset_selection_criteria

User defined subset criteria that defines the columns and rows that the user wants selected.

Commands: 
Get Subset From Criteria

Execute the selection ode and generate a subset dataframe.

Clear

Clear the current criteria selection.

Return

Exit the subset definition.

Help

Subset Criteria Help.

Run Criteria Exceptions

If an exception occurs while executing the user defined criteria the exception is trapped, reported and the user can reenter the criteria.



Run Criteria Empty Resultant df

If executing the user defined criteria results in a df with 0 rows the resultant df is ignored and the user can reenter the criteria.



Process Dataframe Subset

Once you define subset criteria and generate the subset df you need to define wether to save the current subset, process another iteration of the subset You can inspect the subset df before deciding wether to continue generating new subsets or not.


Parameters:
*input_dataframe_title

Current dfc dataframe title to be used in next iteration.

*column_names_list

The list of columns to keep or drop from the input_dataframe.

input_dataframe_column_names

The list of columns to select from to keep or drop.

action_for_columns_in_column_names_list

Boolean flag to keep or drop columns in column_names_list.

Commands: 
Save Subset And Exit

Save the current subset as a dfc subset and exit the subset utility.

Get Next Subset

Use input values defined in form to define criteria and generate another subset.

Inspect Subset Dataframe

Inspect the current subset dataframe in the Data Inspection Chapter.

Open Subset In Excel

Open the current subset df as an excel spreadsheet.

Return

Exit the subset definition.

Help

Subset Criteria Help.

Run Saved Subset


You can rerun a saved subset aequence in order to generate a new copy of the subset. Just click on the Run Saved Subset Sequence button on the main subset console.

Select Subset Sequence To Run



Parameters:
*subset_sequence_title

Subset sequence to run.

*subset_sequence_run_method

The run method for running the subset sequence.

Commands: 
Run Subset Sequence

Run the subset sequence selected in subset_sequence_title.

Delete Subset Sequence

Delete the subset sequence selected in subset_sequence_title.

Return

Exit to subset main console.

Help

Run Subset Manual_Run_Complete_Sequence Help.

Auto Run Method

The Auto Run method will load and run the selected subset sequence without any user intervantion and produce the correct susbset df that can then be used within dfc if desired.

Once the auto run is complete you can review the resultant df and determine to include it as a dfc df.



Commands: 
Add Subset df to dfc dfs

Save the current subset sequence df as a dfc df and make it visible in dfc.

Inspect Subset Dataframe

Inspect the current subset dataframe in the Data Inspection Chapter.

Open Subset In Excel

Open the current subset df as an excel spreadsheet.

Return

Exit the subset definition.

Help

Subset Criteria Help.

Manual Run Method

The Manual Run method will load and run the selected subset sequence one step at a time and allow the user to modify steps as it runs.

Manual Run Step Define Columns

The Manual Run method displays the Define Columns screen and the user can run it with the stored values or change them as desired.


Parameters:
*input_dataframe

dataframe to start step with. Always output df from previous step or initial df.

*column_names_list

List of columns from input_dataframe to keep or drop.

*action_for_columns_in_column_names_list

Keep or Drop columns in columns_names_list.

Commands: 
Get Criteria

Get the criteri for the current step.

Return

Exit the subset sequence run.

Help

Subset Manual Run Help.

Manual Run Step Define Rows

The Manual Run method displays the Define Criteria screen and the user can run it with the stored values or change them as desired.


Parameters:
*subset_dataframe_title

Descriptive title of df generated via current step. This title is used to identify the df to dfc.This title is used as the df title for input to the next step

*subset_selection_criteria

Boolean logic to define the rows of the current subset sequence.

Commands: 
Run Criteria

Run the criteria for the current step to generate a df.

Return

Exit the subset sequence run.

Help

Subset Manual Run Help.

Manual Run Change Step

The Manual Run method allows you to change step parameters as the sequence is run.

Manual Run Original Stored Step Parms

To run the step with the original stored parms do not change anything in the form.


Manual Run Modified Step Parms

Modifying the step parms will result in a different df generated than in the one generated by the stored parms.


Manual Run Sequence Complete

Once all steps in the sequence are complete you can then save the resultant df and sequence.



Commands: 
Save Subset df

Save the subset df as a dfc df and save Sequence.

Add New Step

Add a new step to the sequence.

Inspect Subset Dataframe

Inspect the subset dataframe using data inspection.

Open Subset in Excel

Open the subset df as an excel spreadsheet.

Return

Exit the subset sequence run.

Help

Subset Manual Run Help.

Manual Run Save Subset Sequence df

When you decide to save the resultant df you can select what forms to save.


Parameters:
*dfc_dataframe_name_to_save_final_subset_dataframe_to

Descriptive title of df generated via current step. This title is used to identify the df to dfc.This title is used as the df title for input to the next step

*csv_file_name_to_save_final_subset_dataframe_to

csv file name to save resultant df to.

*subset_sequence_name

Subset sequence name to store subset sequence to for later retrieval.

Commands: 
Save Subset dataframe

Save the resultant df and/or sequence as defined by input values.

Return

Exit the subset sequence run.

Help

Subset Manual Run Help.

Manual Run Add New Step

After running the subset sequence manually you can add a new step to the sequence.



You can now add a new step to the sequence resulting in a new resultamt df.

The input df for the new step is the resultant df from the sequence.

You can continue to new steps like creating a subset sequence.