Modern computerized testing requires a large pool of test items that have to be updated and expanded continuously. To this end, automatic item generation has been recognized as a promising solution, such as the template based techniques and natural language processing based techniques. However, current technologies can hardly deliver high-quality items that own desired psychometric properties and are well aligned with test standards. In this work, we propose a novel trustworthy AI system for automatic item generation and assessment, leveraging the recent advances in generative artificial intelligence, large foundation models, and multi-agent systems. To the best of our knowledge, the proposed system is the first automated pipeline for item generation, psychometric analysis, and quality control. Besides, the proposed system offers user-friendly interfaces for item writers and test developers, serving as an intelligent test development assistant to significantly expedite the test development process.