I have a pandas dataframe with mixed type columns, and I’d like to apply sklearn’s min_max_scaler to some of the columns. Ideally, I’d like to do these transformations in place, but haven’t figured out a way to do that yet. I’ve written the following code that works: import pandas as pd import numpy as np … Read more
I want to find rows that contain a string, like so: DF[DF.col.str.contains(“foo”)] However, this fails because some elements are NaN: ValueError: cannot index with vector containing NA / NaN values So I resort to the obfuscated DF[DF.col.notnull()][DF.col.dropna().str.contains(“foo”)] Is there a better way? 6 Answers 6
I’m new to python and pandas. I’m trying to get a tsv file loaded into a pandas DataFrame. This is what I’m trying and the error I’m getting: >>> df1 = DataFrame(csv.reader(open(‘c:/~/trainSetRel3.txt’), delimiter=”\t”)) Traceback (most recent call last): File “<pyshell#28>”, line 1, in <module> df1 = DataFrame(csv.reader(open(‘c:/~/trainSetRel3.txt’), delimiter=”\t”)) File “C:\Python27\lib\site-packages\pandas\core\frame.py”, line 318, in __init__ raise … Read more
I’m reading in a csv file with multiple datetime columns. I’d need to set the data types upon reading in the file, but datetimes appear to be a problem. For instance: headers = [‘col1’, ‘col2’, ‘col3’, ‘col4’] dtypes = [‘datetime’, ‘datetime’, ‘str’, ‘float’] pd.read_csv(file, sep=’\t’, header=None, names=headers, dtype=dtypes) When run gives a error: TypeError: data … Read more
I have two data frames df1 and df2, where df2 is a subset of df1. How do I get a new data frame (df3) which is the difference between the two data frames? In other word, a data frame that has all the rows/columns in df1 that are not in df2? 15 Answers 15
I have a pandas dataframe. I want to print the unique values of one of its columns in ascending order. This is how I am doing it: import pandas as pd df = pd.DataFrame({‘A’:[1,1,3,2,6,2,8]}) a = df[‘A’].unique() print a.sort() The problem is that I am getting a None for the output. 8 Answers 8
Today I was positively surprised by the fact that while reading data from a data file (for example) pandas is able to recognize types of values: df = pandas.read_csv(‘test.dat’, delimiter=r”\s+”, names=[‘col1′,’col2′,’col3’]) For example it can be checked in this way: for i, r in df.iterrows(): print type(r[‘col1’]), type(r[‘col2’]), type(r[‘col3’]) In particular integer, floats and strings … Read more
I’m trying to read a csv-file from given URL, using Python 3.x: import pandas as pd import requests url = “https://github.com/cs109/2014_data/blob/master/countries.csv” s = requests.get(url).content c = pd.read_csv(s) I have the following error “Expected file path name or file-like object, got <class ‘bytes’> type” How can I fix this? I’m using Python 3.4 6 Answers 6
I know this is a very basic question but for some reason I can’t find an answer. How can I get the index of certain element of a Series in python pandas? (first occurrence would suffice) I.e., I’d like something like: import pandas as pd myseries = pd.Series([1,4,0,7,5], index=[0,1,2,3,4]) print myseries.find(7) # should output 3 … Read more
So I completely understand how to use resample, but the documentation does not do a good job explaining the options. So most options in the resample function are pretty straight forward except for these two: rule : the offset string or object representing target conversion how : string, method for down- or re-sampling, default to … Read more
