数据库21mysql导出导入带header的csv
部分程序需要借助mysql批量导入导出功能,完成csv和数据表之间的转化
导出no header
mysql扩展命令
mysql spj -uroot -p123456 -e “select * from s;” > C:\backup\s.txt
echo "select * from table into outfile '/tmp/table.csv' fields terminated by ',' escaped by '\\' lines terminated by '\n';"|/usr/bin/mysql -uroot database outfile
SELECT * FROM test INTO OUTFILE "E:\\test\\a.csv" FIELDS TERMINATED BY "," ENCLOSED BY ''
LINES TERMINATED BY '\n' mysqldump
mysqldump -u root -phello886 -t -T ~/tmp/ db_name table_name --fields-terminated-by=','
-T制定目录
-w 增加条件判断语句
--result-file=file_name, -r file_name
直接输出到指定文件中。该选项应该用在使用回车换行的系统上(例如:DOS,Windows)。
该选项确保只有一行被使用。
示例:mysqldump -uroot -p --host=localhost --all-databases --result-file=/tmp/mysqldump_result_file.txt
导出with header
基本思路:
SELECT '用户ID', '姓氏', '名字', '电子邮箱'
UNION ALL
SELECT id, first_name, last_name,email
FROM kalacloud_users
INTO OUTFILE '/tmp/kalacloud_users_out_c.csv'
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY '\n';思路01:批量sql中实现以上sql的拼接
参考:mysql导入/导出csv(带列名):https://blog.csdn.net/weixin_43116971/article/details/123634869
核心代码:
SET @table_name = "a";
SET @schema_name = "mydb";
SET @output_name = CONCAT("C://",@table_name,".csv");
SET @cols = NULL;
select GROUP_CONCAT(CONCAT("'",COLUMN_NAME,"'") ORDER BY ordinal_position SEPARATOR ",") INTO @cols
from INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = @table_name AND TABLE_SCHEMA = @schema_name;
SET @sql = CONCAT(" SELECT * FROM (SELECT ", @cols, " UNION ALL SELECT * FROM ", @table_name, ") as r ",
" INTO OUTFILE '", @output_name,
"' FIELDS TERMINATED BY '\\t' ");
PREPARE stmt FROM @sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;思路02:shell脚本方式,将查询结果写入1.log之后,将在1.log的分隔符改为逗号,这样就是常规csv格式。
参考:利用Shell将MySQL数据表导出为csv文件:https://codeantenna.com/a/fuVXuGy2Wp
核心代码:
# write the table into 1.log file
$MYSQL -u root -p****** >1.log << EOF #1.log中是tab分割的sql结果
$statement
EOF
#cat the 1.log file and convert it to csv file
cat 1.log | while read line
do
echo $line | tr " " "," #sql结果被转为逗号分割的csv
done > $database"_"$table.csv由于整个过程在shell中执行,可以较好的接受外部入参。
执行sql文件
mysqldump -uroot -p dbname > dbname .sql
mysql -u用户名 -p密码 数据库名 < 数据库名.sql
mysql -h10.253.29.10 -uroot -p密码 -e"source /opt/rh/test.sql"或者进行mysql交互终端后执行source
mysql>source /home/xxxx/dbname.sql; 导入no header
load data infile
load data infile 'C:\\Users\\UserName\\Desktop\\test.csv'
into table `table`
fields terminated by ',' optionally enclosed by '"' escaped by '"'
lines terminated by '\n';
echo "load data infile "/tmp/table.csv" replace into table table fields escaped by '\\' terminated by ',' lines terminated by '\n';"|/usr/bin/mysql -uroot databasemysqlimport
mysqlimport -L -uroot -proot db01 table_name.txt;
--fields-terminated-by=字符串:设置字符串为字段之间的分隔符,可以为单个或多个字符。默认值为制表符“\t”。
-L, --local:表示从客户端任意路径读取文件导入表中,未设置该选项时,默认只从datadir下同名数据库目录下读取文件导入
--ignore-lines=n:表示可以忽略前n行。
-l, --lock-tables:写入时锁定所有表
-p, --password[=name]:指定用户密码
-u, --user=name:指定登入MySQL用户名
-h, --host=name:指定远程连接的服务器
-c, --columns=name:往表里导入指定字段,如:--columns='Name,Age,Gender'
-C, --compress:在客户端和服务器之间启用压缩传递所有信息
--replace --replace和--ignore选项控制对唯一键值重复现有行的输入行的处理
用法:--replace, -r导入with header
基本思路:
LOAD DATA INFILE 'C:/ProgramData/MySQL/MySQL Server 8.0/Uploads/address_book.csv'
INTO TABLE address_book
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY '\r\n'
IGNORE 1 ROWS;
load data infile 'student.csv' into table student
FIELDS TERMINATED BY ',' -- csv字段分隔符
ENCLOSED BY '"'
LINES TERMINATED BY '\n' -- csv行分隔符
IGNORE 1 ROWS -- 去除表头
(@col1,@col2,@col3,@col4) -- @colN 代表csv文件的第N列的值
set name=@col1,class=@col2,description=@col3,score=@col4; -- 每个列分别赋值
LOAD DATA INFILE 'C:\\Users\\Administrator\\Desktop\\ky\\grxx.csv'
INTO TABLE `grxx` #要插入的表
CHARACTER SET utf8 #设置表编码
FIELDS TERMINATED BY ',' #字段以,分隔
OPTIONALLY ENCLOSED BY '"'
ESCAPED BY '"' #字段以"包围
LINES TERMINATED BY '\r\n' #换行符表示一行结束
IGNORE 1 LINES #忽略第一行
(`index`, `ybh`, `idnum`, `name`, `sex`, `birthday`, `companycode`, `companyname`, `phone`, `dept`); #csv文件中数据按此顺序插入表中
不确定第一行列名忽略后,是否按照列次序导入,还是会自动做自动映射
如果导入时需要做数据格式转换则
LOAD DATA INFILE 'C:/ProgramData/MySQL/MySQL Server 8.0/Uploads/address_book.csv'
INTO TABLE address_book
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY '\r\n'
IGNORE 1 ROWS
(Mobile,@Date,Address)
SET Date = STR_TO_DATE(@Date, '%m/%d/%Y');借助python实现
导入导出csv
CSV文件用python导入mysql:https://blog.csdn.net/qq_15821487/article/details/125487634
参考
利用Shell将MySQL数据表导出为csv文件:https://codeantenna.com/a/fuVXuGy2Wp 如何在 MySQL 中导入和导出 CSV / Excel 文件:https://zhuanlan.zhihu.com/p/444937414 MySQL 导入CSV文件:https://www.lidihuo.com/mysql/mysql-import-csv-file-in-database.html MySQL将表导出为CSV:https://www.yiibai.com/mysql/export-table-to-csv.html
