Skip to content

Commit 891e570

Browse files
committed
Merge branch 'develop'
2 parents ef61097 + 11615ad commit 891e570

File tree

42 files changed

+1029
-351
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

42 files changed

+1029
-351
lines changed

.travis.yml

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
language: java
2+
jdk:
3+
- openjdk8
4+
sudo: false
5+
script: "mvn cobertura:cobertura"
6+
after_success:
7+
- bash <(curl -s https://codecov.io/bash)

README.md

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
1-
# Spring Boot 并发数据聚合库
1+
# Spring Boot 并行数据聚合库
22

3+
[![Build Status](https://travis-ci.org/lvyahui8/spring-boot-data-aggregator.svg?branch=develop)](https://travis-ci.org/lvyahui8/spring-boot-data-aggregator)
4+
[![Codecov](https://codecov.io/gh/lvyahui8/spring-boot-data-aggregator/branch/develop/graph/badge.svg)](https://codecov.io/gh/lvyahui8/spring-boot-data-aggregator/branch/develop)
35
[![License](https://img.shields.io/badge/license-Apache%202-blue.svg)](https://www.apache.org/licenses/LICENSE-2.0)
46
[![Maven Central](https://maven-badges.herokuapp.com/maven-central/io.github.lvyahui8/spring-boot-data-aggregator-starter/badge.svg)](https://maven-badges.herokuapp.com/maven-central/io.github.lvyahui8/spring-boot-data-aggregator-starter)
57
[![GitHub release](https://img.shields.io/github/release/lvyahui8/spring-boot-data-aggregator.svg)](https://github.com/lvyahui8/spring-boot-data-aggregator/releases)
@@ -8,9 +10,9 @@
810

911
在开发后台接口时, 为了开发效率, 我们往往习惯编写串行执行的代码, 去调用不同的接口, 即使这些接口之间并无依赖, 这使得最后开发的接口性能低下, 且数据不方便复用
1012

11-
**此框架目的旨在保持开发效率的同时, 很方便地支持并发和数据复用**
13+
**此框架目的旨在保持开发效率的同时, 很方便地支持并行和数据复用**
1214

13-
当然, 在极端高并发的场景下, 并行调用接口对性能提升并不明显, 但不代表这个项目没有价值. 因为互联网世界的大部分应用, 并不会有非常高的并发访问量
15+
当然, 在极端高并发的场景下,CPU很可能已经跑满, 并行调用接口对性能提升并不明显, 但不代表这个项目没有价值. 因为互联网世界的大部分应用, 并不会有非常高的并发访问量
1416

1517
## 特性
1618

@@ -40,7 +42,7 @@
4042

4143
## 使用方法
4244

43-
### 配置
45+
### 1. 配置
4446

4547
pom.xml
4648

@@ -59,13 +61,13 @@ application.properties
5961
io.github.lvyahui8.spring.base-packages=io.github.lvyahui8.spring.example
6062
```
6163

62-
### 添加注解
64+
### 2. 添加注解
6365

6466
- `@DataProvider` 定义数据提供者
6567
- `@DataConsumer` 定义方法参数依赖类型为其他接口返回值, 其他接口是一个`@DataProvider`
6668
- `@InvokeParameter` 定义方法参数依赖类型为用户输入值
6769

68-
### 查询
70+
### 3. 查询
6971

7072
Spring Bean `DataBeanAggregateQueryFacade` 查询指定的数据的门面
7173

@@ -112,6 +114,8 @@ DataBeanAggregateQueryFacade dataBeanAggregateQueryFacade;
112114

113115
#### 方式一: 函数式调用
114116

117+
注意这里不能将函数式调用改为Lambda表达式, 两者的实际行为是不一致的.
118+
115119
```java
116120
User user = dataBeanAggregateQueryFacade.get(
117121
Collections.singletonMap("userId", 1L),

README_EN.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
# Spring Boot Data Parallel Aggregation Library
22

3+
[![Build Status](https://travis-ci.org/lvyahui8/spring-boot-data-aggregator.svg?branch=develop)](https://travis-ci.org/lvyahui8/spring-boot-data-aggregator)
4+
[![Codecov](https://codecov.io/gh/lvyahui8/spring-boot-data-aggregator/branch/develop/graph/badge.svg)](https://codecov.io/gh/lvyahui8/spring-boot-data-aggregator/branch/develop)
35
[![License](https://img.shields.io/badge/license-Apache%202-blue.svg)](https://www.apache.org/licenses/LICENSE-2.0)
46
[![Maven Central](https://maven-badges.herokuapp.com/maven-central/io.github.lvyahui8/spring-boot-data-aggregator-starter/badge.svg)](https://maven-badges.herokuapp.com/maven-central/io.github.lvyahui8/spring-boot-data-aggregator-starter)
57
[![GitHub release](https://img.shields.io/github/release/lvyahui8/spring-boot-data-aggregator.svg)](https://github.com/lvyahui8/spring-boot-data-aggregator/releases)
@@ -41,7 +43,7 @@ Of course, in an extremely high concurrent scenario, the parallel call interface
4143

4244
## Getting Started
4345

44-
### Configuration
46+
### 1. Configuration
4547

4648
pom.xml
4749

@@ -60,15 +62,15 @@ application.properties
6062
io.github.lvyahui8.spring.base-packages=io.github.lvyahui8.spring.example
6163
```
6264

63-
### Annotation
65+
### 2. Annotation
6466

6567
- `@DataProvider`: define the data provider
6668

6769
- `@DataConsumer`: define the method parameter dependency type as return the value of other interfaces, the other interface is a `@DataProvider`
6870

6971
- `@InvokeParameter`: define the method parameter dependency type as the user input value
7072

71-
### Query
73+
### 3. Query
7274

7375
Spring Bean `dataBeanAggregateQueryFacade` query data facade API
7476

pom.xml

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ limitations under the License.
2020

2121
<groupId>io.github.lvyahui8</groupId>
2222
<artifactId>spring-boot-data-aggregator</artifactId>
23-
<version>1.0.5</version>
23+
<version>1.1.3-SNAPSHOT</version>
2424
<modules>
2525
<module>spring-boot-data-aggregator-core</module>
2626
<module>spring-boot-data-aggregator-autoconfigure</module>
@@ -223,5 +223,19 @@ limitations under the License.
223223
</plugin>
224224
</plugins>
225225
</pluginManagement>
226+
<plugins>
227+
<plugin>
228+
<groupId>org.codehaus.mojo</groupId>
229+
<artifactId>cobertura-maven-plugin</artifactId>
230+
<version>2.7</version>
231+
<configuration>
232+
<formats>
233+
<format>html</format>
234+
<format>xml</format>
235+
</formats>
236+
<check />
237+
</configuration>
238+
</plugin>
239+
</plugins>
226240
</build>
227241
</project>

spring-boot-data-aggregator-autoconfigure/pom.xml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
<parent>
66
<artifactId>spring-boot-data-aggregator</artifactId>
77
<groupId>io.github.lvyahui8</groupId>
8-
<version>1.0.5</version>
8+
<version>1.1.3-SNAPSHOT</version>
99
</parent>
1010
<modelVersion>4.0.0</modelVersion>
1111

spring-boot-data-aggregator-autoconfigure/src/main/java/io/github/lvyahui8/spring/autoconfigure/BeanAggregateAutoConfiguration.java

Lines changed: 117 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -3,36 +3,44 @@
33
import io.github.lvyahui8.spring.aggregate.config.RuntimeSettings;
44
import io.github.lvyahui8.spring.aggregate.facade.DataBeanAggregateQueryFacade;
55
import io.github.lvyahui8.spring.aggregate.facade.impl.DataBeanAggregateQueryFacadeImpl;
6+
import io.github.lvyahui8.spring.aggregate.interceptor.AggregateQueryInterceptor;
7+
import io.github.lvyahui8.spring.aggregate.interceptor.AggregateQueryInterceptorChain;
8+
import io.github.lvyahui8.spring.aggregate.interceptor.impl.AggregateQueryInterceptorChainImpl;
9+
import io.github.lvyahui8.spring.aggregate.model.DataConsumeDefinition;
610
import io.github.lvyahui8.spring.aggregate.model.DataProvideDefinition;
711
import io.github.lvyahui8.spring.aggregate.repository.DataProviderRepository;
812
import io.github.lvyahui8.spring.aggregate.repository.impl.DataProviderRepositoryImpl;
9-
import io.github.lvyahui8.spring.aggregate.service.DataBeanAggregateQueryService;
10-
import io.github.lvyahui8.spring.aggregate.service.impl.DataBeanAggregateQueryServiceImpl;
13+
import io.github.lvyahui8.spring.aggregate.service.DataBeanAggregateService;
14+
import io.github.lvyahui8.spring.aggregate.service.impl.DataBeanAggregateServiceImpl;
1115
import io.github.lvyahui8.spring.aggregate.util.DefinitionUtils;
1216
import io.github.lvyahui8.spring.annotation.DataProvider;
1317
import lombok.extern.slf4j.Slf4j;
18+
import org.apache.commons.lang3.StringUtils;
1419
import org.reflections.Reflections;
1520
import org.reflections.scanners.MethodAnnotationsScanner;
1621
import org.springframework.beans.BeansException;
1722
import org.springframework.beans.factory.annotation.Autowired;
23+
import org.springframework.beans.factory.annotation.Qualifier;
1824
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean;
1925
import org.springframework.boot.context.properties.EnableConfigurationProperties;
2026
import org.springframework.context.ApplicationContext;
2127
import org.springframework.context.ApplicationContextAware;
2228
import org.springframework.context.annotation.Bean;
2329
import org.springframework.context.annotation.Configuration;
30+
import org.springframework.core.Ordered;
2431
import org.springframework.core.annotation.AnnotationUtils;
32+
import org.springframework.core.annotation.Order;
2533
import org.springframework.scheduling.concurrent.CustomizableThreadFactory;
2634
import org.springframework.util.Assert;
27-
import org.springframework.util.StringUtils;
2835

2936
import java.lang.reflect.Method;
3037
import java.lang.reflect.Modifier;
31-
import java.util.Set;
38+
import java.util.*;
3239
import java.util.concurrent.ExecutorService;
3340
import java.util.concurrent.LinkedBlockingDeque;
3441
import java.util.concurrent.ThreadPoolExecutor;
3542
import java.util.concurrent.TimeUnit;
43+
import java.util.stream.Collectors;
3644

3745
/**
3846
* @author lvyahui (lvyahui8@gmail.com,lvyahui8@126.com)
@@ -54,24 +62,90 @@ public void setApplicationContext(ApplicationContext applicationContext) throws
5462

5563
@Bean
5664
@ConditionalOnMissingBean
57-
public DataBeanAggregateQueryFacade dataBeanAggregateQueryFacade() {
58-
return new DataBeanAggregateQueryFacadeImpl(dataBeanAggregateQueryService());
65+
public DataBeanAggregateQueryFacade dataBeanAggregateQueryFacade(
66+
@Qualifier("dataProviderRepository") DataProviderRepository dataProviderRepository) {
67+
return new DataBeanAggregateQueryFacadeImpl(dataBeanAggregateQueryService(dataProviderRepository));
68+
}
69+
70+
private void checkCycle(Map<String,Set<String>> graphAdjMap) {
71+
Map<String,Integer> visitStatusMap = new HashMap<>(graphAdjMap.size() * 2);
72+
for (Map.Entry<String, Set<String>> item : graphAdjMap.entrySet()) {
73+
if (visitStatusMap.containsKey(item.getKey())) {
74+
continue;
75+
}
76+
dfs(graphAdjMap,visitStatusMap,item.getKey());
77+
}
78+
}
79+
80+
private void dfs(Map<String,Set<String>> graphAdjMap,Map<String,Integer> visitStatusMap, String node) {
81+
if (visitStatusMap.containsKey(node)) {
82+
if(visitStatusMap.get(node) == 1) {
83+
List<String> relatedNodes = new ArrayList<>();
84+
for (Map.Entry<String,Integer> item : visitStatusMap.entrySet()) {
85+
if (item.getValue() == 1) {
86+
relatedNodes.add(item.getKey());
87+
}
88+
}
89+
throw new IllegalStateException("There are loops in the dependency graph. Related nodes:" + StringUtils.join(relatedNodes));
90+
}
91+
return ;
92+
}
93+
visitStatusMap.put(node,1);
94+
log.info("visited:{}", node);
95+
for (String relateNode : graphAdjMap.get(node)) {
96+
dfs(graphAdjMap,visitStatusMap,relateNode);
97+
}
98+
visitStatusMap.put(node,2);
5999
}
60100

61101
@Bean
62102
@ConditionalOnMissingBean
63-
public DataBeanAggregateQueryService dataBeanAggregateQueryService () {
64-
DataProviderRepository repository = new DataProviderRepositoryImpl();
65-
scanProviders(repository);
66-
DataBeanAggregateQueryServiceImpl service = new DataBeanAggregateQueryServiceImpl(repository);
67-
service.setRuntimeSettings(createRuntimeSettings());
103+
public DataBeanAggregateService dataBeanAggregateQueryService (
104+
@Qualifier("dataProviderRepository") DataProviderRepository dataProviderRepository) {
105+
if(properties.getBasePackages() != null) {
106+
Map<String,Set<String>> provideDependMap = new HashMap<>(64);
107+
for (String basePackage : properties.getBasePackages()) {
108+
Reflections reflections = new Reflections(basePackage, new MethodAnnotationsScanner());
109+
Set<Method> providerMethods = reflections.getMethodsAnnotatedWith(DataProvider.class);
110+
for (Method method : providerMethods) {
111+
DataProvider beanProvider = AnnotationUtils.findAnnotation(method, DataProvider.class);
112+
@SuppressWarnings("ConstantConditions")
113+
String dataId = beanProvider.id();
114+
Assert.isTrue(Modifier.isPublic(method.getModifiers()),"data provider method must be public");
115+
Assert.isTrue(! StringUtils.isEmpty(dataId),"data id must be not null!");
116+
DataProvideDefinition provider = DefinitionUtils.getProvideDefinition(method);
117+
118+
provider.setId(dataId);
119+
provider.setIdempotent(beanProvider.idempotent());
120+
provider.setTimeout(beanProvider.timeout() > 0 ? beanProvider.timeout() : properties.getDefaultTimeout());
121+
Assert.isTrue(! dataProviderRepository.contains(dataId), "Data providers with the same name are not allowed. dataId: " + dataId);
122+
provideDependMap.put(dataId,provider.getDepends().stream().map(DataConsumeDefinition::getId).collect(Collectors.toSet()));
123+
dataProviderRepository.put(provider);
124+
}
125+
}
126+
checkCycle(provideDependMap);
127+
}
128+
129+
DataBeanAggregateServiceImpl service = new DataBeanAggregateServiceImpl();
130+
RuntimeSettings runtimeSettings = new RuntimeSettings();
131+
runtimeSettings.setIgnoreException(properties.isIgnoreException());
132+
runtimeSettings.setTimeout(properties.getDefaultTimeout());
133+
service.setRepository(dataProviderRepository);
134+
service.setRuntimeSettings(runtimeSettings);
68135
service.setExecutorService(aggregateExecutorService());
136+
service.setInterceptorChain(aggregateQueryInterceptorChain());
137+
service.setTaskWrapperClazz(properties.getTaskWrapperClass());
69138
service.setApplicationContext(applicationContext);
70139
return service;
71140
}
72141

142+
/**
143+
* 允许用户自定义线程池
144+
*
145+
* @return 线程池服务
146+
*/
73147
@Bean(name = "aggregateExecutorService")
74-
@ConditionalOnMissingBean(name = "aggregateExecutorService")
148+
@ConditionalOnMissingBean(name = "aggregateExecutorService",value=ExecutorService.class)
75149
public ExecutorService aggregateExecutorService() {
76150
return new ThreadPoolExecutor(
77151
properties.getThreadNumber(),
@@ -81,39 +155,39 @@ public ExecutorService aggregateExecutorService() {
81155
new CustomizableThreadFactory(properties.getThreadPrefix()));
82156
}
83157

84-
private void scanProviders(DataProviderRepository repository) {
85-
if(properties.getBasePackages() != null) {
86-
for (String basePackage : properties.getBasePackages()) {
87-
Reflections reflections = new Reflections(basePackage, new MethodAnnotationsScanner());
88-
Set<Method> providerMethods = reflections.getMethodsAnnotatedWith(DataProvider.class);
89-
for (Method method : providerMethods) {
90-
dealProviderMethod(repository, method);
91-
}
92-
}
93-
}
94-
}
95-
96-
private void dealProviderMethod(DataProviderRepository repository, Method method) {
97-
DataProvider beanProvider = AnnotationUtils.findAnnotation(method, DataProvider.class);
98-
@SuppressWarnings("ConstantConditions")
99-
String dataId = beanProvider.id();
100-
Assert.isTrue(Modifier.isPublic(method.getModifiers()),"data provider method must be public");
101-
Assert.isTrue(! StringUtils.isEmpty(dataId),"data id must be not null!");
102-
DataProvideDefinition provider = DefinitionUtils.getProvideDefinition(method);
103-
provider.setId(dataId);
104-
provider.setIdempotent(beanProvider.idempotent());
105-
provider.setTimeout(beanProvider.timeout() > 0 ? beanProvider.timeout() : properties.getDefaultTimeout());
106-
repository.put(dataId,provider);
158+
/**
159+
* 允许用户自定义provider存储
160+
*
161+
* @return provider数据仓库
162+
*/
163+
@Bean(name = "dataProviderRepository")
164+
@ConditionalOnMissingBean(DataProviderRepository.class)
165+
public DataProviderRepository dataProviderRepository() {
166+
return new DataProviderRepositoryImpl();
107167
}
108168

109169

110-
private RuntimeSettings createRuntimeSettings() {
111-
RuntimeSettings runtimeSettings = new RuntimeSettings();
112-
runtimeSettings.setEnableLogging(properties.getEnableLogging() != null
113-
? properties.getEnableLogging() : false);
114-
runtimeSettings.setIgnoreException(properties.isIgnoreException());
115-
runtimeSettings.setTimeout(properties.getDefaultTimeout());
116-
return runtimeSettings;
170+
@Bean(name = "aggregateQueryInterceptorChain")
171+
@ConditionalOnMissingBean(AggregateQueryInterceptorChain.class)
172+
public AggregateQueryInterceptorChain aggregateQueryInterceptorChain() {
173+
Map<String, AggregateQueryInterceptor> interceptorMap = applicationContext.getBeansOfType(AggregateQueryInterceptor.class);
174+
AggregateQueryInterceptorChainImpl interceptorChain = new AggregateQueryInterceptorChainImpl();
175+
if(interceptorMap != null && ! interceptorMap.isEmpty()) {
176+
List<AggregateQueryInterceptor> interceptors = new ArrayList<>(interceptorMap.values());
177+
interceptors.sort(new Comparator<AggregateQueryInterceptor>() {
178+
@Override
179+
public int compare(AggregateQueryInterceptor o1, AggregateQueryInterceptor o2) {
180+
Order order1 = o1.getClass().getAnnotation(Order.class);
181+
Order order2 = o2.getClass().getAnnotation(Order.class);
182+
int oi1 = order1 == null ? Ordered.LOWEST_PRECEDENCE : order1.value();
183+
int oi2 = order2 == null ? Ordered.LOWEST_PRECEDENCE : order2.value();
184+
return oi1 - oi2;
185+
}
186+
});
187+
for (AggregateQueryInterceptor interceptor : interceptors) {
188+
interceptorChain.addInterceptor(interceptor);
189+
}
190+
}
191+
return interceptorChain;
117192
}
118-
119193
}

0 commit comments

Comments
 (0)