Data Processing Articles - Page 3 of 20

Practical Use of Python Standard Library: File Traversal and Query

2026-05-25 by boardor

IntroductionIn our daily research work, we often need to handle massive datasets, especially in data-intensive fields such as deep learning, meteorology, and bioinformatics. This demand is particularly prominent. These datasets typically have highly structured naming conventions, a large number of files, and considerable individual file sizes—such as remote sensing images stored in time series, large-scale … Read more

Calculation of Seasonal Extreme Precipitation Indices

2026-05-25 by boardor

Calculation of Seasonal Extreme Precipitation Indices Seasonal Extreme Precipitation Indices Author: Zhi Li Email:[email protected] Author: Xianjun Xiang Email: [email protected] 1 Initialization Initialization %% Initialization clc; % Clear command window clear; % Clear workspace variables close all % Close all figure windows % Define time range StartYear = 1961; % Starting year of data EndYear = … Read more

21.6k Stars! Parsing GBs of JSON Data Per Second, SIMD Instructions Boost Performance by 25 Times!

2026-05-24 by boardor

Yesterday afternoon, my boss approached me with a stern expression: “Why is this data processing so slow? The clients are complaining.” I glanced at the screen and noticed that the progress bar was almost stuck in place, leaving me speechless. I believe most people have had similar experiences; even when the code logic is flawless, … Read more

Step-by-Step: How to Conveniently Use Python and Pandas for Data Anonymization

2026-05-22 by boardor

Produced by Big Data Digest Compiled by: Yihang, Hu Jia, Aileen Recently, I received a dataset containing sensitive information about customers that should never be disclosed under any circumstances. The dataset is located on one of our servers, a relatively secure place. However, I wanted to copy the data to my local disk for easier … Read more

Quick Start Guide to MATLAB: Part 09

2026-05-21 by boardor

This series serves as a learning note for a quick start with MATLAB, aiming to document the learning process for future reference. I am honored if it can help you. Practicing the Feynman learning method, I openly share and welcome communication and corrections! 📚 Complete Resource Map Title Core Skills Application Scenarios Quick Start to … Read more

kevinsr: A Handy Python Tool for Data Processing!

2026-05-20 by boardor

When processing data, do you find the steps of organizing and analyzing cumbersome? The kevinsr module is a “handy tool” for Python data processing, simplifying operations like data cleaning and statistics, thereby enhancing efficiency. Whether it’s students working on data analysis assignments or programmers handling project data, it can be of great assistance. This article … Read more

Simple Python Code for Office Automation: Copy and Use

2026-05-20 by boardor

The following are commonly used code scenarios and core library recommendations for Python in office automation, organized based on multiple practical cases: 1. Excel Data Processing Data Cleaning and Merging import pandas as pd def merge_excel(folder_path): all_data = [] for file in Path(folder_path).glob('*.xlsx'): df = pd.read_excel(file, skiprows=2) df['Source'] = file.name all_data.append(df) return pd.concat(all_data).dropna().to_excel("merged.xlsx") Features: Automatically … Read more

Continuous Monitoring of Temperature Sensors in Wireless Networks Using MATLAB and XBee: A Study with MATLAB Code

2026-05-20 by boardor

✅ Author Profile: A research enthusiast and MATLAB simulation developer, skilled in data processing, modeling simulation, program design, complete code acquisition, paper reproduction, and scientific simulation. 🍎 Previous reviews, follow the personal homepage: MATLAB Research Studio 🍊 Personal motto: Investigate to gain knowledge, complete MATLAB code and simulation consultation available via private message. 🔥 Content … Read more

MATLAB T-S Diagram

2026-05-18 by boardor

Author: The Eighth Galaxy – Xiang Xianjun Email: [email protected] Data Source: www.argo.org.cn File Import close all clear clc %% File Import file_S = "E:\data\woa18_decav_s02_04.nc"; % Salinity file_T = "E:\data\woa18_decav_t02_04.nc"; % Temperature % Information View ncdisp(file_S); % Salinity ncdisp(file_T); % Temperature % Variable Extraction lon = ncread(file_S, 'lon'); % Longitude latitude = ncread(file_S, 'lat'); % Latitude … Read more

How to Build a Custom Modbus Communication Block Instruction

2026-05-16 by boardor

1. Introduction Modbus, as a widely used general communication protocol, is extensively applied in frequency converters, smart instruments, and various intelligent devices. Previous articles have shared application cases based on the development of custom Modbus communication block instructions, specifically including the communication program implementation between S7-200SMART and Siemens V20 frequency converter, as well as Omron … Read more