Spring Batch 之 Sample(复合格式文件的读、多文件的写)(七)

长平狐 发布于 2012/08/27 15:48
阅读 1K+
收藏 1

      前面关于Spring Batch的文章,讲述了SpringBatch对CSV文件的读写操作对XML文件的操作,以及对固定长格式文件的操作。这些事例,同一个Reader读取的都是相同格式的数据,最终写入一个文件。如果遇到下面这样的数据,并想将学生信息和商品信息分类后写入两个文件,应该如何处理呢?

student,200001,ZhangSan,18,78
goodsPNH001011000200.1zhangshana2011/12/18 01:12:36
student,200002,LiSi,19,79
goodsPNH001022000300.1zhangshanb2011/12/19 01:12:36
student,200003,WangWu,20,80
goodsPNH001033000400.1zhangshanc2011/12/20 01:12:36

  * 以student开头的数据代表学生信息,以goods开头代表商品信息

这次将和大家一起探讨Spring Batch读取复合格式的数据,然后写入不同的文件的处理方式。

      工程结构如下图:

       applicationContext.xml和log4j.xml前文已经叙述过,在此不做赘述。

      本实例的核心配置文件batch.mxl内容如下:

1 <? xml version="1.0" encoding="UTF-8" ?>
2 < bean:beans xmlns ="http://www.springframework.org/schema/batch"
3 xmlns:bean ="http://www.springframework.org/schema/beans" xmlns:xsi ="http://www.w3.org/2001/XMLSchema-instance"
4 xmlns:p ="http://www.springframework.org/schema/p" xmlns:tx ="http://www.springframework.org/schema/tx"
5 xmlns:aop ="http://www.springframework.org/schema/aop" xmlns:context ="http://www.springframework.org/schema/context"
6 xmlns:util ="http://www.springframework.org/schema/util"
7 xsi:schemaLocation ="http://www.springframework.org/schema/beans
8 http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
9 http://www.springframework.org/schema/tx
10 http://www.springframework.org/schema/tx/spring-tx-3.0.xsd
11 http://www.springframework.org/schema/aop
12 http://www.springframework.org/schema/aop/spring-aop-3.0.xsd
13 http://www.springframework.org/schema/context
14 http://www.springframework.org/schema/context/spring-context-2.5.xsd
15 http://www.springframework.org/schema/batch
16 http://www.springframework.org/schema/batch/spring-batch-2.1.xsd
17 http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd" >
18
19 < bean:import resource ="applicationContext.xml" />
20 <!-- Job的配置信息 -->
21 < job id ="multiTypeSingleFileJob" >
22 < step id ="xmlFileReadAndWriterStep" >
23 < tasklet >
24 < chunk reader ="multiTypesItemReader" writer ="multiTypesItemWriter"
25 commit-interval ="1" >
26 < streams >
27 < stream ref ="studentWriter" />
28 < stream ref ="goodsWriter" />
29 </ streams >
30 </ chunk >
31 </ tasklet >
32 </ step >
33 </ job >
34
35 <!-- 不同格式数据的文件读取 -->
36 < bean:bean id ="multiTypesItemReader"
37 class ="org.springframework.batch.item.file.FlatFileItemReader" scope ="step" >
38 < bean:property name ="resource"
39 value ="file:#{jobParameters['inputFilePath']}" />
40 < bean:property name ="lineMapper" >
41 < bean:bean
42 class ="org.springframework.batch.item.file.mapping.PatternMatchingCompositeLineMapper" >
43 < bean:property name ="tokenizers" >
44 < bean:map >
45 < bean:entry key ="student*" value-ref ="studentTokenizer" />
46 < bean:entry key ="goods*" value-ref ="goodsTokenizer" />
47 </ bean:map >
48 </ bean:property >
49 < bean:property name ="fieldSetMappers" >
50 < bean:map >
51 < bean:entry key ="student*" value-ref ="studentFieldSetMapper" />
52 < bean:entry key ="goods*" value-ref ="goodsFieldSetMapper" />
53 </ bean:map >
54 </ bean:property >
55 </ bean:bean >
56 </ bean:property >
57 </ bean:bean >
58 < bean:bean id ="studentTokenizer"
59 class ="org.springframework.batch.item.file.transform.DelimitedLineTokenizer" >
60 < bean:property name ="delimiter" value ="," />
61 < bean:property name ="names" >
62 < bean:list >
63 < bean:value >student </ bean:value >
64 < bean:value >ID </ bean:value >
65 < bean:value >name </ bean:value >
66 < bean:value >age </ bean:value >
67 < bean:value >score </ bean:value >
68 </ bean:list >
69 </ bean:property >
70 </ bean:bean >
71 < bean:bean id ="studentFieldSetMapper"
72 class ="org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper" >
73 < bean:property name ="prototypeBeanName" value ="student" />
74 < bean:property name ="distanceLimit" value ="100" />
75 </ bean:bean >
76 <!-- 学生Pojo类 -->
77 < bean:bean id ="student"
78 class ="com.wanggc.springbatch.sample.multitypessinglefile.pojo.Student"
79 scope ="prototype" />
80
81 < bean:bean id ="goodsTokenizer"
82 class ="org.springframework.batch.item.file.transform.FixedLengthTokenizer" >
83 < bean:property name ="columns" value ="6-13,14-17,18-22,23-32,33-" />
84 < bean:property name ="names"
85 value ="isin,quantity,price,customer,buyDay" />
86 </ bean:bean >
87 < bean:bean id ="goodsFieldSetMapper"
88 class ="org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper" >
89 < bean:property name ="prototypeBeanName" value ="goods" />
90 </ bean:bean >
91 <!-- 商品Pojo类 -->
92 < bean:bean id ="goods"
93 class ="com.wanggc.springbatch.sample.multitypessinglefile.pojo.Goods"
94 scope ="prototype" />
95
96 < bean:bean id ="multiTypesItemWriter"
97 class ="com.wanggc.springbatch.sample.multitypessinglefile.MultiItemWriter" >
98 < bean:property name ="delegates" >
99 < bean:list >
100 < bean:ref bean ="studentWriter" />
101 < bean:ref bean ="goodsWriter" />
102 </ bean:list >
103 </ bean:property >
104 </ bean:bean >
105 <!-- 学生信息的写 -->
106 < bean:bean id ="studentWriter"
107 class ="org.springframework.batch.item.file.FlatFileItemWriter" scope ="step" >
108 < bean:property name ="resource"
109 value ="file:#{jobParameters['outputFilePathStudent']}" />
110 < bean:property name ="lineAggregator" >
111 < bean:bean
112 class ="org.springframework.batch.item.file.transform.FormatterLineAggregator" >
113 < bean:property name ="fieldExtractor" >
114 < bean:bean
115 class ="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor" >
116 < bean:property name ="names" value ="ID,name,age,score" />
117 </ bean:bean >
118 </ bean:property >
119 < bean:property name ="format" value ="%-9s%-9s%3d%-2.0f" />
120 </ bean:bean >
121 </ bean:property >
122 </ bean:bean >
123 <!-- 商品信息的写 -->
124 < bean:bean id ="goodsWriter"
125 class ="org.springframework.batch.item.file.FlatFileItemWriter" scope ="step" >
126 < bean:property name ="resource"
127 value ="file:#{jobParameters['outputFilePathGoods']}" />
128 < bean:property name ="lineAggregator" >
129 < bean:bean
130 class ="org.springframework.batch.item.file.transform.DelimitedLineAggregator" >
131 < bean:property name ="fieldExtractor" >
132 < bean:bean
133 class ="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor" >
134 < bean:property name ="names"
135 value ="isin,quantity,price,customer,buyDay" />
136 </ bean:bean >
137 </ bean:property >
138 </ bean:bean >
139 </ bean:property >
140 </ bean:bean >
141 </ bean:beans >

       21-33行配置了Job的基本信息。  

      36-57行配置了Reader的基本信息。FlatFileItemReader的lineMapper属性使用SpringBatch核心类PatternMatchingCompositeLineMapper的时候,会将读取的记录按照不同的方式映射成我们的Pojo对象。当然首先我们要配置不同的tokenizers(43-48)和fieldSetMappers(49-54),并告诉它当前的记录按照那条原则去解析和映射。如45行所示,我们指定key为student*的时候,用studentTokenizer去解析成fieldset,用studentFieldSetMapper将studentTokenizer解析好的fieldset记录映射成Student对象。我们指定的key,其实也就是student开头的记录,*是通配符。PatternMatchingCompositeLineMapper支持两种通配符:*和?,前者代表多个字符,后者仅代表一个字符。至于student和goods信息如何映射成pojo对象,前面的文章中已经做过详细的介绍,这里就不做赘述了。

      96-104行配置了Writer的基本信息。Writer也是使用代理的方式,学生信息使用106-122行定义的studentWriter按照固定长的格式写入学生信息文件中,商品信息使用124-141行定义的goodsWriter按照CSV的格式写入商品信息文件中。MultiItemWriter的代码很简单,就不做详细解释了。如下:

package com.wanggc.springbatch.sample.multitypessinglefile;

import java.util.ArrayList;
import java.util.List;

import org.springframework.batch.item.ItemWriter;

import com.wanggc.springbatch.sample.multitypessinglefile.pojo.Goods;
import com.wanggc.springbatch.sample.multitypessinglefile.pojo.Student;

/**
* 写处理类。
*
*
@author Wanggc
*
*
@param <T>
*/
@SupssWarnings("unchecked")
public class MultiItemWriter<T> implements ItemWriter<T> {
/** 写代理 */
private List<ItemWriter<? super T>> delegates;

public void setDelegates(List<ItemWriter<? super T>> delegates) {
this.delegates = delegates;
}

@Override
public void write(List<? extends T> items) throws Exception {
// 学生信息的Writer
ItemWriter studentWriter = (ItemWriter) delegates.get(0);
// 商品信息的Writer
ItemWriter goodsWriter = (ItemWriter) delegates.get(1);
// 学生信息
List<Student> studentList = new ArrayList<Student>();
// 商品信息
List<Goods> goodsList = new ArrayList<Goods>();
// 将传过来的信息按照不同的类型添加到不同的List中
for ( int i = 0; i < items.size(); i++) {
if ("Student".equals(items.get(i).getClass().getSimpleName())) {
studentList.add((Student) items.get(i));
} else {
goodsList.add((Goods) items.get(i));
}
}
// 如果学生List中有数据,就执行学生信息的写
if (studentList.size() > 0) {
studentWriter.write(studentList);
}
// 如果商品List中有数据,就执行商品信息的写
if (goodsList.size() > 0) {
goodsWriter.write(goodsList);
}
}
}

      至此,复合文件的读写操作已经讨论结束了。注意实例没有配置Processor。下面是一些辅助文件的信息。

      student和goods类的信息与前面文章一样,就不再贴出代码了。

      Job启动的代码如下:

package com.wanggc.springbatch.sample.multitypessinglefile;

import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobParametersBuilder;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;

public class Launch {
public static void main(String[] args) {
ApplicationContext context = new ClassPathXmlApplicationContext(
"batch.xml");
JobLauncher launcher = (JobLauncher) context.getBean("jobLauncher");
Job job = (Job) context.getBean("multiTypeSingleFileJob");

try {
// JOB实行
JobExecution result = launcher.run(
job,
new JobParametersBuilder()
.addString("inputFilePath",
"C:\\testData\\multiTypesInput.txt")
.addString("outputFilePathStudent",
"C:\\testData\\student.txt")
.addString("outputFilePathGoods",
"C:\\testData\\goods.csv")
.toJobParameters());
// 运行结果输出
System.out.println(result.toString());
} catch (Exception e) {
e.printStackTrace();
}
}
}

      Input文件内容如下图:

      处理结果的学生信息文件如下图:

     处理结果的商品信息文件如下图:

      Spring Batch对复合格式文件的读写操作就讨论到这里。至此,Spring Batch对文件简单操作的讨论也告一段落,下次将讨论Spring Batch读写DB的操作。

 


原文链接:http://www.cnblogs.com/gulvzhe/archive/2011/12/18/2291570.html
加载中
OSCHINA
登录后可查看更多优质内容
返回顶部
顶部