
The Effect of Category-specific Reviews of NSFC Application Based on Randomized Controlled Trials: Piloting by the Division Ⅲ of Department of Management Sciences
Gang WU, Zhongfei CHEN, Feng WANG, Jian YU
China Journal of Econometrics ›› 2022, Vol. 2 ›› Issue (2) : 228-236.
The Effect of Category-specific Reviews of NSFC Application Based on Randomized Controlled Trials: Piloting by the Division Ⅲ of Department of Management Sciences
This paper uses the randomized controlled trials method to evaluate the pilot effect of the category-specific review in 2021, which is composed of 1, 110 applications in Economic Sciences (G03) of Division Ⅲ in the Department of Management Sciences. These applications cover the General Program and the Young Scientists Funds. We find that category-specific review can effectively increase the consensus of peer-review on the Original Exploratory program, the Cutting-edge program, and Cross-category program. There is significant evidence that the rate for the applications being on the conference review and finally approved in the above three programs have increased. Furthermore, there is no significant difference in the distribution and mean value of the overall scores in the treatment and control groups, which implies that the category-specific review pilot has not systematically affected the results of the peer-review.
category-specific review / NSFC / randomized controlled trials / department of management sciences / effect evaluation {{custom_keyword}} /
表1 实验组与对照组在四类科学问题属性的上会率统计 |
原创 | 前沿 | 需求 | 交叉 | 合计 | |||
2021 | 实验组 | 比例 | 14.29% | 25.00% | 21.94% | 25.00% | 22.28% |
对照组 | 比例 | 16.67% | 20.39% | 21.94% | 10.00% | 20.90% | |
2020 | 未分类 | 比例 | 8.70% | 20.18% | 21.27% | 13.56% | 19.67% |
表2 实验组与对照组在四类科学问题属性的资助率统计 |
原创 | 前沿 | 需求 | 交叉 | 合计 | |
实验组资助率变化 | 5.71% | 7.60% | 2.04% | 10.00% | 3.19% |
同2020年资助率变化 | 1.36% | 4.75% | 0.20% | 6.50% | 1.60% |
注: 实验组资助率变化是2021年实验组项目资助率与同年的对照组项目资助率的差值, 同2020年资助率变化是2021年实验组项目资助率与2020年项目资助率的差值. |
表3 实验组与对照组的评审结果均值统计及 |
实验组 | 对照组 | 均值差值 | |||
总得分 | 3.078 | 3.051 | 0.027 | 0.422 | 0.673 |
综合评价 | 2.480 | 2.456 | 0.024 | 0.674 | 0.501 |
资助建议 | 0.599 | 0.596 | 0.003 | 0.105 | 0.917 |
样本数 | 579 | 531 |
表4 实验组与对照组评审结果的Wilcoxon rank-sum检验 |
学科分类 | 统计量 | 总得分 | 综合评价 | 资助建议 | 样本数 |
G0306国际经济与贸易学科 | 0.106 | 211 | |||
0.914 | 0.916 | 0.804 | |||
G0307金融经济学科 | 381 | ||||
0.890 | 0.853 | 0.972 | |||
G0311农林经济管理学科 | 0.853 | 0.400 | 1.049 | 518 | |
0.394 | 0.689 | 0.294 | |||
总计 | 0.421 | 0.181 | 0.542 | 1110 | |
0.674 | 0.856 | 0.588 |
来源: 作者统计. |
表5 变量定义及其统计描述 |
变量 | 定义 | 样本量 | 均值 | 标准差 | 最小值 | 最大值 |
Score | 总得分 | 1110 | 3.065 | 1.060 | 1 | 6 |
Score2 | 综合评价 | 1110 | 0.597 | 0.483 | 0 | 2 |
Score3 | 资助建议 | 1110 | 2.468 | 0.590 | 1 | 4 |
treat | 1表示实验组, 0表示对照组 | 1110 | 0.522 | 0.500 | 0 | 1 |
age | 年龄 | 1110 | 37.330 | 7.132 | 27 | 68 |
gend | 性别, 1表示男性, 0表示女性 | 1110 | 0.511 | 0.500 | 0 | 1 |
prof | 职称, 1表示正教授, 0表示其他 | 1110 | 0.145 | 0.352 | 0 | 1 |
cat | 项目类型, 1是面上项目, 0是青年项目 | 1110 | 0.402 | 0.490 | 0 | 1 |
univ | 高校类型, 1表示985高校, 0表示其他高校 | 1110 | 0.195 | 0.397 | 0 | 1 |
陈克新, 熊杰, 2020年度无机非金属学科分类评审改革措施及基金项目受理情况分析[J]. 硅酸盐学报, 2021, (1): 1- 6.
{{custom_citation.content}}
{{custom_citation.annotation}}
|
冯玉宽, 宋永喜, 戴洁, 陈怡文, 洪雪辉, 等. 2019年度国家自然科学基金临床肿瘤学科面上项目分类申请与评审情况分析与思考[J]. 中国科学基金, 2021, (1): 122- 128.
{{custom_citation.content}}
{{custom_citation.annotation}}
|
洪永淼, 理解现代计量经济学[J]. 计量经济学报, 2021, (1): 266- 284.
{{custom_citation.content}}
{{custom_citation.annotation}}
|
李静海, 构建新时代科学基金体系夯实世界科技强国根基[J]. 中国科学基金, 2018, (4): 345- 350.
{{custom_citation.content}}
{{custom_citation.annotation}}
|
李静海, 全面深化科学基金改革更好发挥在国家创新体系中的基础引领作用[J]. 中国科学基金, 2019, (3): 209- 214.
{{custom_citation.content}}
{{custom_citation.annotation}}
|
刘哲, 何建军, 郭郁葱, 基于大气科学学科发展特点, 解读项目分类评审改革新举措[J]. 科学通报, 2021, (2): 187- 192.
{{custom_citation.content}}
{{custom_citation.annotation}}
|
{{custom_citation.content}}
{{custom_citation.annotation}}
|
Callaway E, (2019). 'Randomistas' Who Used Controlled Trials to Fight Poverty Win Economics Nobel[N]. Nature. https://www.nature.com/articles/d41586-019-03125-y/.
{{custom_citation.content}}
{{custom_citation.annotation}}
|
{{custom_citation.content}}
{{custom_citation.annotation}}
|
{{custom_citation.content}}
{{custom_citation.annotation}}
|
{{custom_citation.content}}
{{custom_citation.annotation}}
|
{{custom_ref.label}} |
{{custom_citation.content}}
{{custom_citation.annotation}}
|
/
〈 |
|
〉 |