Leaderboard for Large Language Models on Geospatial Code Generation (AutoGEEval & AutoGEEval++)

Rank Model Developer AutoGEEval
Pass@1		 AutoGEEval
Pass@3		 AutoGEEval
Pass@5		 AutoGEEval++
Pass@1		 AutoGEEval++
Pass@3		 AutoGEEval++
Pass@5		 Type

Related Papers

Our research publications

AutoGEEval: A Multimodal and Automated Evaluation Framework for Geospatial Code Generation on GEE with Large Language Models

Huayi Wu, Zhangxiao Shen, Shuyang Hou, Jianyuan Liang, et al.

ISPRS International Journal of Geo-Information, July 2025

Read Paper GitHub

AutoGEEval++: A Multi-Level and Multi-Geospatial-Modality Automated Evaluation Framework for Large Language Models in Geospatial Code Generation on Google Earth Engine

Huayi Wu, Zhangxiao Shen, Shuyang Hou, Haoyue Jiao, et al.

arXiv preprint, June 2025

Read Paper GitHub

GeoGraphRAG: A graph-based retrieval-augmented generation approach for empowering large language models in automated geospatial modeling

Jianyuan Liang, Shuyang Hou, Haoyue Jiao, Yaxian Qing, et al.

International Journal of Applied Earth Observation and Geoinformation, August 2025

Read Paper

Chain-of-programming (CoP): empowering large language models for geospatial code generation task

Shuyang Hou, Haoyue Jiao, Zhangxiao Shen, Jianyuan Liang, et al.

International Journal of Digital Earth, May 2025

Read Paper FigShare

Design and application of a semantic-driven geospatial modeling knowledge graph based on large language models

Jianyuan Liang, Shuyang Hou, Anqi Zhao, Qingyang Xu, et al.

Geo-spatial Information Science, May 2025

Read Paper

GEE-OPs: An operator knowledge base for geospatial code generation on the Google Earth Engine platform powered by large language models

Shuyang Hou, Jianyuan Liang, Anqi Zhao, Huayi Wu

Geo-spatial Information Science, May 2025

Read Paper FigShare

Geo-FuB: A Method for Constructing an Operator-Function Knowledge Base for Geospatial Code Generation with Large Language Models

Shuyang Hou, Anqi Zhao, Jianyuan Liang, Zhangxiao Shen, et al.

Knowledge-Based Systems, April 2025

Read Paper GitHub

GeoCode-GPT: A large language model for geospatial code generation

Shuyang Hou, Zhangxiao Shen, Anqi Zhao, Jianyuan Liang, et al.

International Journal of Applied Earth Observation and Geoinformation, March 2025

Read Paper GitHub

Can large language models generate geospatial code?

Shuyang Hou, Zhangxiao Shen, Jianyuan Liang, Anqi Zhao, et al.

arXiv preprint, October 2024

Read Paper

A GPT-enhanced framework on knowledge extraction and reuse for geographic analysis models in Google Earth Engine

Jianyuan Liang, Anqi Zhao, Shuyang Hou, Fengying Jin, et al.

International Journal of Digital Earth, September 2024

Read Paper