Why do I get 2, 5, 8, 11 in the Triangle column instead of 3, 6, 9, 12 when I use the ChainLadder package? df = pd.DataFrame({ 'origin': ['200101', '200105', '200203', '200204', '200310', '200311'], 'development': ['200103', '200207', '200206', '200301', '200312', '200405'], 'value': [100, 150, 200, 250, 300, 350] }) # Convert DataFrame to Triangle triangle = cl.Triangle(df, origin='origin', development='development', columns='value', cumulative=False) triangle_new = triangle.grain("OQDQ") triangle_new 2 5 8 11 14 17 20 23 26 29 32 35 38 41 2001Q1 100.00...

Replace over 1M pandas values using a dictionary Now, I'm researching about STRING network When I want to replace STRING protein to Entrez ID, I see that STRING data has over 1M pandas rows.(~11M rows) So, I tried to replace STRING protein ID(like 9606.ENSP00000~~) to Entrez ID(integer). All methods was very slow. but when I use this code, the time elapsed < 5 min. is this code not problem? Here is the code. ` def replace_string_id(self, data: pd.DataFrame, target_col: str, rename_dict: dict) -> pd.DataFrame: tmp_list = [] for...

Faster way to subtract columns and update row values? Faster way to subtract columns and update row values? colSet1 = list(range(20)) colSet2 = list(range(21,40)) colSet3 = list(range(41,60)) for col1 in colSet1: for col2 in colSet2: for col3 in colSet3: df[col1+col2+col3+"_newcol"] = df[col1+col2+col3]-df[col1] I tried rsub but it seems to give wrong results, not sure, if i implemented it correctly. What is the correct way to use rsub in this case, If that is the fastest way?

scraping a page: using bs4 and requests [closed] trying to gather the data form the page "https://clutch.co/il/it-services" and that said i - think that there are probably several options to do that a. using bs4 and requests b. using pandas this first approach uses a. import requests from bs4 import BeautifulSoup import pandas as pd url = "https://clutch.co/il/it-services" response = requests.get(url) soup = BeautifulSoup(response.content, "html.parser") company_names = soup.find_all("h3", class_="company-name") locations =...

Unable to convert json to csv for a specific columns in pandas I have a dataset in this form: df.head(2) Employee_Name EID Details Lisa FG546HJ {"Summary": "Worked as a HR professional.", "Services_List": ["HR", "Marketing", "Hiring"],"On_payroll?": false} Martin H5644HH {"Summary": "Worked as a UI Designer.", "Services_List": ["Frontend", "UI", "UX"],"On_payroll?": True} As you can see, the 'Details' column is in json format...But I want the output to be in this form: Employee_Name EID Summary...

Auto update the streamlit app using st.empty and placeholder I want my streamlit app to be automaticaly updated, with data being updated without the need to refresh the browser, I tried ` placeholder1 = st.empty() with placeholder1: Sensor1.metric( label="Sensor 1 RH", value=sensors_df3['Sensor 1'].iloc[-1] ) Sensor2.metric( label="Sensor 2 RH", value=sensors_df3['Sensor 2'].iloc[-1] ) Sensor3.metric( label="Sensor 3 RH", value=sensors_df3['Sensor 3'].iloc[-1] )...

How to plot a grouped bar plot with data from multiple dataframes how to plot multiple categories with different data lengths on the same bar chart in python? Here is my code: import pandas as pd import matplotlib.pyplot as plt Type_X = [200000, 150000] Type_Y = [168000, 120000] Comb = [192000] Cat = ['With Time Constraints', 'With Production Constraints'] Cat2 = ['Optimised Combination of Devices'] df = pd.DataFrame({'Type X': Type_X, 'Type Y': Type_Y,}, index=Cat) df2 = pd.DataFrame({'Comb' : Comb}, index=Cat2) print(df)...

python pandas add a fixed value to get cumulative sum I have a dataframe , I want to add a fixed number 5 to the first element of the dataframe, to get cumulative sum import pandas as pd data = {'Values': [10, 5000, 6000, 7000, 8000, 9000, 8000]} df = pd.DataFrame(data) Expected result like below, the first row is calculate 10 + 5 =15, second row is cumulative sum 15+5 = 20, then 20+5 please...

Plot year by year in the same plot (plotly) I tried plot time series with many year in the same star and end day-month. For example, what I need, is that data from 01/01 for 2018, 2019 etc in the same plot in order to compare different data from different years. The code that I do, plot year by year in subplot, but I would like is a plot in only one plotly. Link of data to download and can run the script <a...

Curve fitting a percentage of data I have two sets of real data, xdata and ydata, and I'm trying to do an exponencial fitting to them, as shown in my code below. def func(x, a, b, c): return a * np.exp(-b * x) + c popt, pcov = curve_fit(func, xdata, ydata) plt.scatter(xdata, ydata, color='r', label="Demanda máxima", alpha=0.5) plt.plot(xdata, func(xdata, *popt), 'r--') The fitting is working fine, however, I need 90% of my data points to be under/over my fitted curve. Is that a way to do this? Thanks in advice.

Applying within group rankings for Pandas DataFrame The problem statement has people choosing 3 items and ranking them 1 to 3. My Pandas DataFrame contains one row for each item selected, where the first column is the person's name and the second column is the ranking the person has given to the selection. Something like: df = pd.DataFrame.from_dict({ 'Name':['Alice','Alice','Alice','Bob','Bob','Bob','Charlie','Charlie','Charlie'], 'Item':[...], 'Rank':[1,2,3,3,1,2,None,None,None] }) The issue is some people did not assign rankings to...

Filtering Motion Data in Python by Maximum Position for each Time Period I am trying to filter data points by time and position in python so that only the highest position for each second of time is left in the data set. So here is the situation, I used a motion sensor to record the oscillations of a spring/mass system and I am trying to do an exponential curve fit of the damping. The highest position decreases over time, and the periods last about a second. I thought maybe if I turned the times into integers then I could get rid of everything except the max...

Pandas: Pandas Rolling Condition to find smallest values and return the matched result I have the Data frame with two set of values A & B in column 'ID'. Need to find the smallest value from previous rows in the same columns 'MT' & 'Values'. If 'MT' & 'Value's column current row value is smaller than its previous values the 'Matched Values' column & 'Date Values' column has to be updated with the conditions that matched. <a href="" rel="nofollow...

How to resolve 'KeyError' issue in Pandas dataframe when using a custom function in a loop? Unable to find out the problem of a defined function that is used in a loop. The program I am trying to develop, is about a supply chain management that involves planing and managing deliveries and warehouse capacities. I have written a function that is working outside the main code, but when I try to use it in the main procedure, it faces with an error. Assume that there is a pandas dataframe called df like below: df = {'Name': {0: 'a', 1: 'c', 2: 'j', 3: 'd', 4: 'e'}, 'Type': {0: 1, 1: 1,...

Retrieving data from an Excel file in Django using Pandas: How can I include duplicates that are being overwritten by Python? I am creating a django application which retrieves data from an excel file. I access the excel file through the scrape.py file: import pandas as pd import os from pathlib import Path BASE_DIR = Path(file).resolve().parent DATA_DIR = os.path.join(BASE_DIR, "Data") database_path = os.path.join(DATA_DIR, "Data.xlsx") Database = pd.read_excel(database_path) Then I have this class: class skimask: def...

Why I can't load dataset from seaborn I am trying to load the dataset iris from seaborn in my spyder My code was simple: import seaborn as sns df = sns.load_dataset('iris') print(df) I am not getting output. I have tried in Jupiter notebook also, not getting any output What to do? I tried to load a dataset form seaborn named: 'iris' and also 'flights' but I can't.

How do I compare two columns in a Pandas DataFrame and output values from other columns based on the match? So I have a pandas dataframe. A: 3, 4, 1, 2, 1, B 1, 2, 3, 4 C Red, Blue, Yellow, Green D Yes, No, Maybe, True what I want to be able to do is sequentially check column A against column B, if there is a match then output C and D relative to B. For example the data frame above would be converted into A Yellow, Green, Red, Blue, Red B Maybe, True, Yes, No, Yes I am rather new to python and pandas so I might be missing something simple here but I cannot think of solutions for this problem and...

The python pandas to_csv function is missing rows when outputting to AWS s3 bucket on Heroku The script is running fine but I get 2 different outputs converting dataframe to csv when ran on Anaconda Jupyter notebook on windows vs on Heroku. The issue seems to be related to index but cannot figure it out. The csv index column from Heroku jumps from 10 to 433 (see below). The version generated by Jupyter is displaying correctly at first glance. code I am using to send write dataframe to csv and send to s3 bucket: s3 = s3fs.S3FileSystem(anon=False, key=config.AWS_ACCESS_KEY_ID,...

How can I delete a row from a Pandas dataframe based on user input of the row's index? I'm trying to remove a row from a dataframe, but instead of specifying the index i want the user to be able to input the row's index, and then it deletes the row based on that value rowdel = input("enter row # to delete") df.drop([rowdel], axis=0, inplace=True) This is my code, but i get the error "['x'] not found in axis" (x being the input)

Frequency of one specific value in a DataFrame column I am attempting to count the frequency of one specific value 'Devonport' in the column 'Ground' in a data set. I have tried to use the Groupby function and the value.counts() and it is only returning the first and last 5 unique values in the column which has a total of 173 unique values. I know there is one occurrence of 'Devonport' in the column 'Ground' because I opened the csv file and filtered on the column, but I am unable to get the function to return this in jupyter notebook. Any help...

My code to concatenate several csv files is running very slow, how to optimize it? I have 200 csv files in a folder, with around 3000 lines each. I have this code to select the folder using a dialog, then it scans for files that match specific prefix and suffix. The final objective is to concatenate all files to a xlsx file, but the csv files do not have all the same number of columns. In that case, the columns should be kept as broad as possible (new columns should be added, with blank info when it does not exist for other files/lines). When I run the code below for 2...

How can I change the legends of two-axis plot? How to improve my two-axis plot? How can I get this two-DF, secondary axis plot to show me complete information? For example: the legend of the plot only shows "V" and "V (right)" which are the names of the respective DF, but I want to show the correct names I gave to each DF. The code: reservas = pd.DataFrame(reservas_json[-5000:]) reservas.set_index('d', inplace=True, drop=True) date = 'd' usd_of = pd.DataFrame(usd_of_json[-5000:]) usd_of.set_index('d', inplace=True, drop=True) date = 'd'...

what is going on with these results. These are not about osquery.

This has been an ongoing bug with SO integration for some time now

SO seems to have an issue where their rss feeds for a tag spew garbage occasionally. I filed it, and many people said they see it was well. It’s been happening for years. I doubt SO cares. :(

osquery file events giving events for un-monitored directory on Mac os Trying to capture file events on mac os OS: Monterey osquery table used: es_process_file_events Flag used: --disable_endpointsecurity_fim=false I am monitoring two directories /tmp and /usr but am not getting any file event from this folder surprisingly I am getting events from /Libraray/Application\ Support which even I am not monitoring. Has anyone faced similar issue? please help?

Slack invite link is not working The invite link given on the website and everywhere else is not working. It says This link is no longer active. Can anyone share a valid link here? submitted by /u/nanikarthik14 [link] [comments]

Hrm. ^^^ is unfortunate. I’ve pinged slack about

Made a new one in https://github.com/osquery/osquery-site/pull/279

system policy: java(4297) deny(1) file-write-mode on mac os In my Project I am trying to install one install4j build there osquery.app also gets installed. OS:- Mac os Ventura In fresh install osquery.app gets installed properly do not get any error in my folder .But when I try upgrade scenario osquery.app in console gives this error system policy: java(4297) deny(1) file-write-mode /Library/Application Support/test/osquery.app Tried SIP disable it works there.But now I want it to work in SIP enable. Can anyone help? I am not understanding is it...