当前位置:首页 > 开发 > 系统架构 > 架构 > 正文

Spring Batch: Intro II -- Read-Write-Process for Derby

发表于: 2014-06-20   作者:DavyJones2010   来源:转载   浏览次数:
摘要: 1. In the following example, we will read data from multiple csv files and then write to Derby DB.     1) In order to read from multiple csv, we need "org.springframework.batch.item.fi

1. In the following example, we will read data from multiple csv files and then write to Derby DB.

    1) In order to read from multiple csv, we need "org.springframework.batch.item.file.MultiResourceItemReader" as reader.

        And config flatFileReader for delegate which reading single csv file.

    2) In order to write to Derby, we need "org.springframework.batch.item.database.JdbcBatchItemWriter" as writer.

        And config derbyDataSource as datasource.

 

2.

    1) pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>edu.xmu.spring.batch</groupId>
	<artifactId>spring-batch</artifactId>
	<version>0.0.1-SNAPSHOT</version>

	<properties>
		<spring.version>3.2.2.RELEASE</spring.version>
		<spring.batch.version>2.2.0.RELEASE</spring.batch.version>
		<junit.version>4.11</junit.version>
	</properties>

	<dependencies>
		<!-- Spring Core -->
		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-core</artifactId>
			<version>${spring.version}</version>
		</dependency>
		<!-- Spring jdbc, for database -->
		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-jdbc</artifactId>
			<version>${spring.version}</version>
		</dependency>
		<!-- Derby database driver -->
		<dependency>
			<groupId>org.apache.derby</groupId>
			<artifactId>derby</artifactId>
			<version>10.10.1.1</version>
		</dependency>
		<!-- Spring Batch dependencies -->
		<dependency>
			<groupId>org.springframework.batch</groupId>
			<artifactId>spring-batch-core</artifactId>
			<version>${spring.batch.version}</version>
		</dependency>
		<dependency>
			<groupId>org.springframework.batch</groupId>
			<artifactId>spring-batch-infrastructure</artifactId>
			<version>${spring.batch.version}</version>
		</dependency>
	</dependencies>
</project>

   2) domain.ddl 

DROP TABLE ROOT.DOMAIN;
CREATE TABLE ROOT.DOMAIN(ID INT NOT NULL, USERNAME VARCHAR(100), PASSWORD VARCHAR(100), DATE_ADDED DATE);

   3) Domain.java

package edu.xmu.spring.batch.model;

import java.util.Date;

public class Domain {
    int id;
    String username;
    String password;
    Date dateAdded;

    public int getId() {
	return id;
    }

    public void setId(int id) {
	this.id = id;
    }

    public String getUsername() {
	return username;
    }

    public void setUsername(String username) {
	this.username = username;
    }

    public String getPassword() {
	return password;
    }

    public void setPassword(String password) {
	this.password = password;
    }

    public Date getDateAdded() {
	return dateAdded;
    }

    public void setDateAdded(Date dateAdded) {
	this.dateAdded = dateAdded;
    }

}

    4) context.xml

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://www.springframework.org/schema/beans 
		http://www.springframework.org/schema/beans/spring-beans-3.2.xsd">

	<import resource="database.xml" />
	<import resource="../jobs/spring-batch-job-derby.xml" />

	<bean id="jobLauncher"
		class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
		<property name="jobRepository" ref="jobRepository" />
	</bean>

	<bean id="jobRepository"
		class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean">
		<property name="transactionManager" ref="transactionManager" />
	</bean>
	
</beans>

    5) database.xml

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:jdbc="http://www.springframework.org/schema/jdbc" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://www.springframework.org/schema/beans 
		http://www.springframework.org/schema/beans/spring-beans-3.2.xsd
		http://www.springframework.org/schema/jdbc
		http://www.springframework.org/schema/jdbc/spring-jdbc-3.2.xsd">

	<!-- connect to database -->
	<bean id="derbyDataSource"
		class="org.springframework.jdbc.datasource.DriverManagerDataSource">
		<property name="driverClassName" value="org.apache.derby.jdbc.EmbeddedDriver" />
		<property name="url"
			value="jdbc:derby:C:/Documents and Settings/******/MyDB" />
		<property name="username" value="******" />
		<property name="password" value="******" />
	</bean>
	<bean id="transactionManager"
		class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />
</beans>

    6) spring-batch-job-derby.xml

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:batch="http://www.springframework.org/schema/batch" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="
		http://www.springframework.org/schema/beans 
		http://www.springframework.org/schema/beans/spring-beans-3.2.xsd
		http://www.springframework.org/schema/batch
		http://www.springframework.org/schema/batch/spring-batch-2.2.xsd">

	<bean id="domain" class="edu.xmu.spring.batch.model.Domain" scope="prototype"/>

	<batch:job id="readMultiFileJob">
		<batch:step id="readAndWriteCsvFile">
			<batch:tasklet>
				<batch:chunk reader="multiResourceReader" writer="derbyItemWriter"
					processor="flatFileItemProcessor" commit-interval="10" />
			</batch:tasklet>
		</batch:step>
	</batch:job>

	<bean id="multiResourceReader"
		class="org.springframework.batch.item.file.MultiResourceItemReader">
		<property name="resources" value="file:csv/input/domain-*.csv" />
		<property name="delegate" ref="flatFileItemReader" />
	</bean>
	<bean id="flatFileItemReader" class="org.springframework.batch.item.file.FlatFileItemReader">
		<property name="lineMapper">
			<bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
				<property name="lineTokenizer">
					<bean
						class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
						<property name="names" value="id, username, password" />
					</bean>
				</property>
				<property name="fieldSetMapper">
					<bean
						class="org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper">
						<property name="prototypeBeanName" value="domain" />
					</bean>
				</property>
			</bean>
		</property>
	</bean>

	<bean id="flatFileItemProcessor" class="edu.xmu.spring.batch.processor.CustomCSVItemProcessor" />

	<bean id="derbyItemWriter"
		class="org.springframework.batch.item.database.JdbcBatchItemWriter">
		<property name="dataSource" ref="derbyDataSource" />
		<property name="sql">
			<value>
            <![CDATA[        
            	INSERT INTO ROOT.DOMAIN(ID, USERNAME, PASSWORD, DATE_ADDED) 
				VALUES (:id, :username, :password, :dateAdded)
            ]]>
			</value>
		</property>
		<property name="itemSqlParameterSourceProvider">
			<bean
				class="org.springframework.batch.item.database.BeanPropertyItemSqlParameterSourceProvider" />
		</property>
	</bean>

</beans>

    7) CustomCSVItemProcessor.java

package edu.xmu.spring.batch.processor;

import java.util.Calendar;
import java.util.Date;

import org.springframework.batch.item.ItemProcessor;

import edu.xmu.spring.batch.model.Domain;

public class CustomCSVItemProcessor implements ItemProcessor<Domain, Domain> {

    @Override
    public Domain process(Domain item) throws Exception {
	Date date = Calendar.getInstance().getTime();
	item.setUsername(item.getUsername().replace(',', '-')
		.replaceAll(" ", ""));
	item.setPassword(item.getPassword().replace(',', '-'));
	item.setDateAdded(date);
	return item;
    }

}

   8) App.java

package edu.xmu.spring.batch;

import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.context.support.ClassPathXmlApplicationContext;

public class App {
    public static void main(String[] args) {

	ClassPathXmlApplicationContext context = new ClassPathXmlApplicationContext(
		new String[] { "spring/batch/config/context.xml" });

	JobLauncher jobLauncher = (JobLauncher) context.getBean("jobLauncher");
	Job job = (Job) context.getBean("readMultiFileJob");

	JobExecution execution;
	try {
	    execution = jobLauncher.run(job, new JobParameters());
	    System.out.println("Exit Status : " + execution.getStatus());
	} catch (Exception e) {
	    e.printStackTrace();
	}

	System.out.println("Done");

	context.close();
    }
}

 

Reference Links:

1) http://www.mkyong.com/spring-batch/spring-batch-example-csv-file-to-database/

 

 

Spring Batch: Intro II -- Read-Write-Process for Derby

  • 0

    开心

    开心

  • 0

    板砖

    板砖

  • 0

    感动

    感动

  • 0

    有用

    有用

  • 0

    疑问

    疑问

  • 0

    难过

    难过

  • 0

    无聊

    无聊

  • 0

    震惊

    震惊

编辑推荐
1. Intro to Spring Batch 1) Spring Batch is a framework for Batch Processing - Execution of a
Spring Batch 之 Spring Batch 简介(一) Spring Batch是一个轻量级的,完全面向Spring的批处理框架
Spring Batch Concepts(2) Spring Batch supportsthe following database engines: DB2, Derby, H2,
转http://www.coderli.com/spring-batch-intro-sample/ 官方文档地址:http://docs.spring.io/sprin
转载请注明出处: http://renjie120.iteye.com/ 用过ibatis之后,感觉hibernate太复杂. 使用过derby
Spring Batch Concepts Chapter The below figure shows two kinds of Spring Batch components:inf
Spring Batch Documentation: http://static.springsource.org/spring-batch/reference/index.html
Spring Batch Documentation: http://static.springsource.org/spring-batch/reference/index.html
Spring Batch Documentation: http://static.springsource.org/spring-batch/reference/index.html
Spring Batch Concepts Chapter The below figure shows two kinds of Spring Batch components:inf
版权所有 IT知识库 CopyRight © 2009-2015 IT知识库 IT610.com , All Rights Reserved. 京ICP备09083238号