facat/bidding_watcher
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: 更新README以反映投标监控系统项目详情

Browse Source
This commit is contained in:
dmy
2026-01-13 21:32:59 +08:00
parent e410053ddd
commit 2fcfb452ec
1 changed files with 249 additions and 153 deletions
Download Patch File Download Diff File Expand all files Collapse all files

402
README.md
View File

@@ -1,106 +1,88 @@
<p align="center">
<a href="http://nestjs.com/" target="blank"><img src="https://nestjs.com/img/logo-small.svg" width="120" alt="Nest Logo" /></a>
</p>
# 投标信息智能监控系统
[circleci-image]: https://img.shields.io/circleci/build/github/nestjs/nest/master?token=abc123def456
[circleci-url]: https://circleci.com/gh/nestjs/nest
一个基于 TypeScript 的 Web 应用，用于自动爬取商务投标平台的最新信息，将符合条件的投标项目突出显示，为用户提供精准的投标信息监控服务。
<p align="center">A progressive <a href="http://nodejs.org" target="_blank">Node.js</a> framework for building efficient and scalable server-side applications.</p>
<p align="center">
<a href="https://www.npmjs.com/~nestjscore" target="_blank"><img src="https://img.shields.io/npm/v/@nestjs/core.svg" alt="NPM Version" /></a>
<a href="https://www.npmjs.com/~nestjscore" target="_blank"><img src="https://img.shields.io/npm/l/@nestjs/core.svg" alt="Package License" /></a>
<a href="https://www.npmjs.com/~nestjscore" target="_blank"><img src="https://img.shields.io/npm/dm/@nestjs/common.svg" alt="NPM Downloads" /></a>
<a href="https://circleci.com/gh/nestjs/nest" target="_blank"><img src="https://img.shields.io/circleci/build/github/nestjs/nest/master" alt="CircleCI" /></a>
<a href="https://discord.gg/G7Qnnhy" target="_blank"><img src="https://img.shields.io/badge/discord-online-brightgreen.svg" alt="Discord"/></a>
<a href="https://opencollective.com/nest#backer" target="_blank"><img src="https://opencollective.com/nest/backers/badge.svg" alt="Backers on Open Collective" /></a>
<a href="https://opencollective.com/nest#sponsor" target="_blank"><img src="https://opencollective.com/nest/sponsors/badge.svg" alt="Sponsors on Open Collective" /></a>
<a href="https://paypal.me/kamilmysliwiec" target="_blank"><img src="https://img.shields.io/badge/Donate-PayPal-ff3f59.svg" alt="Donate us"/></a>
<a href="https://opencollective.com/nest#sponsor" target="_blank"><img src="https://img.shields.io/badge/Support%20us-Open%20Collective-41B883.svg" alt="Support us"></a>
<a href="https://twitter.com/nestframework" target="_blank"><img src="https://img.shields.io/twitter/follow/nestframework.svg?style=social&label=Follow" alt="Follow us on Twitter"></a>
</p>
<!--[![Backers on Open Collective](https://opencollective.com/nest/backers/badge.svg)](https://opencollective.com/nest#backer)
[![Sponsors on Open Collective](https://opencollective.com/nest/sponsors/badge.svg)](https://opencollective.com/nest#sponsor)-->
## 技术栈
## Description
### 后端
- **框架**: NestJS
- **语言**: TypeScript
- **数据库**: PostgreSQL (TypeORM)
- **爬虫**: axios
- **任务调度**: @nestjs/schedule
- **AI 服务**: OpenAI API
[Nest](https://github.com/nestjs/nest) framework TypeScript starter repository.
### 前端
- **框架**: Vue.js 3
- **构建工具**: Vite
- **UI**: Tailwind CSS
- **状态管理**: Pinia
## Project setup
## 项目结构
```bash
$ npm install
```
src/
├── ai/ # AI 模块
│ ├── ai.controller.ts
│ ├── ai.service.ts
│ ├── Prompt.ts
│ └── entities/
├── bids/ # 投标业务模块
│ ├── controllers/
│ ├── entities/
│ └── services/
├── crawler/ # 爬虫模块
│ ├── services/
│ │ ├── bid-crawler.service.ts
│ │ ├── cdt_target.ts
│ │ ├── chng_target.ts
│ │ ├── ceic_target.ts
│ │ ├── cgnpc_target.ts
│ │ ├── chdtp_target.ts
│ │ ├── cnncecp_target.ts
│ │ ├── cnooc_target.ts
│ │ ├── eps_target.ts
│ │ ├── espic_target.ts
│ │ ├── powerbeijing_target.ts
│ │ ├── sdicc_target.ts
│ │ └── szecp_target.ts
│ └── entities/
├── database/ # 数据库模块
├── keywords/ # 关键词管理模块
├── schedule/ # 定时任务
│ └── tasks/
│ └── bid-crawl.task.ts
├── scripts/ # 脚本工具
│ ├── ai-recommendations.ts
│ ├── crawl.ts
│ ├── deploy.ps1
│ ├── remove-duplicates.ts
│ └── update-source.ts
└── common/ # 公共模块
└── logger/
frontend/
└── src/
├── components/
│ ├── Dashboard.vue
│ ├── Dashboard-AI.vue
│ ├── PinnedProject.vue
│ ├── Bids.vue
│ ├── Keywords.vue
│ └── CrawlInfo.vue
├── App.vue
└── main.ts
```
## Compile and run the project
## 快速开始
```bash
# development
$ npm run start
### 1. 环境准备
# watch mode
$ npm run start:dev
确保已安装 Node.js (18+) 和 PostgreSQL。
# production mode
$ npm run start:prod
```
### 2. 数据库配置
## Run tests
```bash
# unit tests
$ npm run test
# e2e tests
$ npm run test:e2e
# test coverage
$ npm run test:cov
```
## Deployment
When you're ready to deploy your NestJS application to production, there are some key steps you can take to ensure it runs as efficiently as possible. Check out the [deployment documentation](https://docs.nestjs.com/deployment) for more information.
If you are looking for a cloud-based platform to deploy your NestJS application, check out [Mau](https://mau.nestjs.com), our official platform for deploying NestJS applications on AWS. Mau makes deployment straightforward and fast, requiring just a few simple steps:
```bash
$ npm install -g @nestjs/mau
$ mau deploy
```
With Mau, you can deploy your application in just a few clicks, allowing you to focus on building features rather than managing infrastructure.
## Resources
Check out a few resources that may come in handy when working with NestJS:
- Visit the [NestJS Documentation](https://docs.nestjs.com) to learn more about the framework.
- For questions and support, please visit our [Discord channel](https://discord.gg/G7Qnnhy).
- To dive deeper and get more hands-on experience, check out our official video [courses](https://courses.nestjs.com/).
- Deploy your application to AWS with the help of [NestJS Mau](https://mau.nestjs.com) in just a few clicks.
- Visualize your application graph and interact with the NestJS application in real-time using [NestJS Devtools](https://devtools.nestjs.com).
- Need help with your project (part-time to full-time)? Check out our official [enterprise support](https://enterprise.nestjs.com).
- To stay in the loop and get updates, follow us on [X](https://x.com/nestframework) and [LinkedIn](https://linkedin.com/company/nestjs).
- Looking for a job, or have a job to offer? Check out our official [Jobs board](https://jobs.nestjs.com).
## Support
Nest is an MIT-licensed open source project. It can grow thanks to the sponsors and support by the amazing backers. If you'd like to join them, please [read more here](https://docs.nestjs.com/support).
## Stay in touch
- Author - [Kamil Myśliwiec](https://twitter.com/kammysliwiec)
- Website - [https://nestjs.com](https://nestjs.com/)
- Twitter - [@nestframework](https://twitter.com/nestframework)
## License
Nest is [MIT licensed](https://github.com/nestjs/nest/blob/master/LICENSE).
## How to Run
### 1. Database Setup
Update the `.env` file with your PostgreSQL credentials:
复制 `.env.example` 为 `.env` 并配置数据库连接：
```env
DATABASE_TYPE=postgres
@@ -110,73 +92,187 @@ DATABASE_USERNAME=your_username
DATABASE_PASSWORD=your_password
DATABASE_NAME=bidding
DATABASE_SYNCHRONIZE=true
```
### 2. Install Dependencies
```bash
npm install
cd frontend && npm install
```
### 3. Build and Start
```bash
# From the root directory
cd frontend && npm run build
cd ..
npm run build
npm run start
```
## Features
### Frontend Features
- **Dashboard**: View high priority bids and today's bids
- **Date Filtering**:
- Click "3天" or "7天" buttons to filter bids from the last 3 or 7 days
- The filter only limits the start date, showing all data from the selected start date onwards (including data newer than the end date)
- **Keyword Filtering**: Filter bids by keywords (saved in localStorage)
- **All Bids**: View all bids with pagination and source filtering
- **Keyword Management**: Add and delete keywords with weight-based priority
### Backend Features
- **Multi-Source Crawling**: Crawls bidding information from multiple sources:
- ChdtpCrawler
- ChngCrawler
- SzecpCrawler
- CdtCrawler
- EpsCrawler
- CnncecpCrawler
- CgnpcCrawler
- CeicCrawler
- EspicCrawler
- PowerbeijingCrawler
- **Automatic Retry**: If a crawler returns 0 items, it will be retried after all crawlers complete
- **Proxy Support**: Configurable proxy settings via environment variables
- **Scheduled Tasks**: Automatic crawling at scheduled intervals
### Environment Variables
```env
# Database
DATABASE_TYPE=postgres
DATABASE_HOST=localhost
DATABASE_PORT=5432
DATABASE_USERNAME=your_username
DATABASE_PASSWORD=your_password
DATABASE_NAME=bidding
DATABASE_SYNCHRONIZE=true
# Proxy (optional)
# 代理配置（可选）
PROXY_HOST=your_proxy_host
PROXY_PORT=your_proxy_port
PROXY_USERNAME=your_proxy_username
PROXY_PASSWORD=your_proxy_password
# AI 配置
OPENAI_API_KEY=your_openai_api_key
```
## Initial Setup
### 3. 安装依赖
The system will automatically initialize with the preset keywords: "山东", "海", "建设", "工程", "采购". You can manage these and view crawled bidding information at http://localhost:3000.
```bash
# 安装后端依赖
npm install
# 安装前端依赖
cd frontend && npm install
```
### 4. 运行项目
```bash
# 开发模式 - 后端
npm run start:dev
# 开发模式 - 前端
cd frontend && npm run dev
```
### 5. 构建生产版本
```bash
# 构建前端
cd frontend && npm run build
# 构建后端
cd .. && npm run build
# 运行生产版本
npm run start:prod
```
## 核心功能
### 智能爬虫模块
- **多源爬取**: 支持 12 个主流招标网站
- 中国大唐集团电子商务平台 (CDT)
- 中国华能集团有限公司电子商务平台 (CHNG)
- 中国南方电网电子商务平台 (CSG)
- 中国海洋石油集团有限公司 (CNOOC)
- 中国华电集团有限公司电子商务平台 (CHDTP)
- 国家能源投资集团有限责任公司 (CNNCECP)
- 中国核工业集团有限公司 (CNNC)
- 中国电力建设集团有限公司 (POWERCHINA)
- 中国能源建设集团有限公司 (CEIC)
- 中国石油天然气集团有限公司 (CNPC)
- 国家电网有限公司 (SGCC)
- 北京电力交易中心 (POWERBEIJING)
- **智能防封策略**:
- 随机请求间隔 (3-8 秒)
- 轮换 User-Agent
- 异常检测与自动重试机制
- 代理支持
- **定时任务**: 每 30 分钟自动执行爬取
### 数据处理与存储
- **数据模型**:
- 投标项目标题
- 详细页面 URL
- 发布时间
- 招标单位
- 截止日期
- 关键词匹配
- 优先级评分
- **增量存储**:
- 通过 URL 哈希值判断是否为新数据
- 仅存储当天和最近 7 天的历史数据
- 每日自动清理 30 天前的数据
### 关键词智能监控
- **预设关键词**: "山东", "海", "建设", "工程", "采购"
- **自定义关键词**: 通过 Web 界面添加/删除关键词
- **权重设置**: 可设置关键词权重 (1-5 级)
- **匹配逻辑**:
- 标题完全匹配和部分匹配
- 多关键词叠加权重
- 支持正则表达式高级匹配
### AI 智能推荐
- **智能分析**: 使用 AI 分析投标信息的相关性
- **推荐评分**: 基于关键词匹配和内容分析生成推荐评分
- **智能摘要**: 自动生成投标信息摘要
### Web 展示界面
- **仪表盘**:
- 高优先级投标信息（匹配自定义关键词）
- 今日新增投标列表（按时间倒序）
- AI 推荐投标信息
- 置顶项目
- **交互功能**:
- 关键词管理面板
- 按日期/来源/关键词筛选
- 信息标记已读/未读状态
- 项目置顶功能
- 爬取信息查看
- **响应式设计**: 适配桌面和移动设备
## API 接口
### 投标信息
- `GET /api/bids` - 获取投标列表（支持分页、筛选）
- `GET /api/bids/high-priority` - 获取高优先级投标
- `GET /api/bids/today` - 获取今日投标
### 关键词管理
- `GET /api/keywords` - 获取所有关键词
- `POST /api/keywords` - 添加新关键词
- `DELETE /api/keywords/:id` - 删除关键词
### AI 服务
- `GET /api/ai/recommendations` - 获取 AI 推荐投标
- `POST /api/ai/analyze` - 分析投标信息
### 爬虫管理
- `GET /api/crawler/info` - 获取爬取信息
- `POST /api/crawler/trigger` - 手动触发爬取
## 前端路由
- `/` - 仪表盘（默认页面）
- `/bids` - 全部投标信息
- `/keywords` - 关键词管理
- `/ai` - AI 推荐页面
- `/crawl-info` - 爬取信息
## 测试
```bash
# 单元测试
npm run test
# E2E 测试
npm run test:e2e
# 测试覆盖率
npm run test:cov
```
## 部署
项目包含 PowerShell 部署脚本 `src/scripts/deploy.ps1`，用于自动化部署流程。
## 环境变量
| 变量名 | 说明 | 默认值 |
|--------|------|--------|
| DATABASE_TYPE | 数据库类型 | postgres |
| DATABASE_HOST | 数据库主机 | localhost |
| DATABASE_PORT | 数据库端口 | 5432 |
| DATABASE_USERNAME | 数据库用户名 | - |
| DATABASE_PASSWORD | 数据库密码 | - |
| DATABASE_NAME | 数据库名称 | bidding |
| DATABASE_SYNCHRONIZE | 自动同步数据库 | true |
| PROXY_HOST | 代理主机 | - |
| PROXY_PORT | 代理端口 | - |
| PROXY_USERNAME | 代理用户名 | - |
| PROXY_PASSWORD | 代理密码 | - |
| OPENAI_API_KEY | OpenAI API 密钥 | - |
## 许可证
MIT