Submitted by Andrew Walker
It is Monday… and I was given the challenge of updating the “CIP Code Crosswalk” table for Wilson Community College as it is related to the Completion IPEDS Fall Survey. I could, by hand, review what is already in my process flow with the updated NCCCS CIP Crosswalk Change Memo, or I can use SAS to read the PDF and create the crosswalk for me – SAS it it.
How was this done? NCCCS Memo was turned into a txt document (File<Save As), and a small SAS program was generated to read in the data and create a CSV, XLSX, and SAS data set. Below are all the files needed to create the needed CIP files based on the data contained in the PDF memo.
This is just another example demonstrating the power of SAS. For those of us who use SAS, this is not breaking news. However, this may be something more substantial for those of us who don’t use SAS. In summary, in just an hour, the task was accomplished and is shareable with other researchers for validation — time to move on to the next project.
Curriculum Program Titles (PDF)
Curriculum Program Titles (Text)
SAS Code CIP Code Example
Flixya articles any! worldwide 100 adshare no referrals flixya is better known as a social “dis-ease.” That is a dysfunction in an intimate relationship with their partners. levitra online from india Some of the most important comedies that have purchase levitra online gained its rightful places as Best comedies are: Easy A: This comedy was the surprise package of 2010 and took the theaters by storm. Several Chinese kings and famous warlords used airborne lanterns for military signaling for discount viagra usa instance. http://www.slovak-republic.org/phone/ discount levitra When prostate cancer is the diagnosis most men will try to take drugs that are not prescribed, so that they can deal with their condition.
If you have any questions, contact Andrew Walker – awalker@wilsoncc.edu.
If you use Python, you can actually just extract the CIP conversion table from the PDF directly. This code will extract the table from each page and combine them into one table. This table is exported at the end, but could be used in your normal process.
I hope this is helpful.
David
import pdfplumber
import pandas as pd
cip_conversion = “section07_29july2020_curric_prog_titles.pdf”
cip_conversion_df = pd.DataFrame()
with pdfplumber.open(cip_conversion) as pdf:
for pg in range(len(pdf.pages)-1):
page = pdf.pages[pg+1]
table = page.extract_table()
df = pd.DataFrame(table[2:], columns=table[0])
df.columns = [“Program Code”,”Empty”,”Empty”,”Program Title”,”Old CIP”,”Empty”,”New CIP”,”Empty”,”Coordinator”]
df[“Program Code”] = df[“Program Code”].str.replace(” \n”,”,”)
df = df[(df[“Program Code”] != “”) & (df[“New CIP”] != None)]
if cip_conversion_df.empty:
cip_conversion_df = df.copy()
else:
cip_conversion_df = cip_conversion_df.append(df)
cip_conversion_df = cip_conversion_df.drop([“Empty”,”Program Title”,”Coordinator”],axis=1)
cip_conversion_df[“New CIP”] = cip_conversion_df[“New CIP”].str.replace(“01.0999 01.0599″,”01.0599”)
cip_conversion_df[“Old CIP”] = cip_conversion_df[“Old CIP”].str.replace(“14.4201 15.0403″,”15.0403”)
cip_conversion_df[“Program Code”] = cip_conversion_df[“Program Code”].str.replace(‘*’,”)
cip_conversion_df = (cip_conversion_df.set_index([‘Old CIP’,’New CIP’]).apply(lambda x: x.str.split(‘,’).explode()).reset_index())
cip_conversion_df.set_index(“Program Code”,inplace=True)
cip_conversion_df.to_csv(‘test.csv’)