基于流程建模的流数据处理及服务化系统
DOI:
作者:
作者单位:

1.北方工业大学 数据工程研究院;2.北方工业大学数据工程研究院

作者简介:

通讯作者:

中图分类号:

基金项目:

国家重点研发计划,公共安全风险防控与应急技术装备(2017YFC0804406)


View-driven flow data real-time processing and service system
Author:
Affiliation:

North China University of Technology

Fund Project:

National Key R&D Plan(No:2017YFC0804406)

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    随着大数据时代的来临,各领域产生的信息飞速增长,其中有些数据具有数据量大,实时到达、持续不间断、变化快等特征,需要被实时处理,被称为流数据。流数据的处理需求也复杂多变,业务人员要进行相应的算法定制,不仅需要相关的编程知识,更要应对繁琐的处理流程和冗长的开发周期。为解决上述问题,本文设计并实现了基于流程建模的流数据处理及服务化系统,提供了对于多源流数据的实时接入,流数据服务化以及流数据处理服务化的能力。该系统将流数据处理过程封装为服务提供给用户,允许用户拖拽组合流数据处理和服务化模块、配置相关参数,来定义流数据处理及服务化的过程,快速又自然地实现流数据处理及服务化的任务,将处理结果经由服务路由实时推送到其它应用系统,满足不同的业务需求。案例分析表明,与传统的流数据处理系统相比,本系统具有高效、灵活、可配置等特点,在实用性、可用性和伸缩性方面都更有优势。

    Abstract:

    With the advent of the era of big data, the information generated in various fields is growing rapidly. Some of them have the characteristics of large amount of data, real-time arrival, continuous uninterrupted, fast change, etc., and need to be processed in real time. The processing requirements of these data are also complex and changeable. The business personnel need to carry out corresponding algorithm customization, which not only requires relevant programming knowledge, but also has to deal with cumbersome processing flow and lengthy development cycle. In order to solve the above problems, this paper designs and implements a stream data processing and service system based on process modeling, which provides real-time access to multi-source stream data, stream data service and stream data processing service. The system encapsulates the stream data processing process into a service provided to the user, allows the user to drag and drop the combined stream data processing and service module, configures relevant parameters, defines the process of stream data processing and service, and implements stream data processing quickly and naturally. And service-oriented tasks, the processing results are pushed to other application systems in real time through service routes to meet different business needs. Case studies show that the system is more efficient, flexible, and configurable than traditional streaming data processing systems, and has advantages in terms of usability, usability, and scalability.

    参考文献
    相似文献
    引证文献
引用本文
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2019-11-27
  • 最后修改日期:2019-12-04
  • 录用日期:2019-12-10
  • 在线发布日期:
  • 出版日期: