Create a test generator that will take in a topic or material and produce a database of multiple choice questions
To design a test generator that creates a database of multiple-choice questions based on a given topic or material, we need to gather detailed requirements for the system. Below are the proposed requirements for inputs, outputs, and logic.
Once the initial requirements are drafted, they should be reviewed by potential users and stakeholders to ensure that they meet their needs and expectations. Based on feedback, the requirements may be refined or expanded.
This is a high-level overview of the requirements for the test generator system. The actual implementation may require additional details and specifications based on the user's specific needs and the technical environment in which the system will be deployed.
To design the architecture of an "agent" system that uses GPT "actors" to model a creative process, let's first define the components and their interactions. The system will be procedural, with creative steps modeled by specialized GPT actors. Here's a conceptual overview:
initialize web interface
initialize session manager
initialize actor manager
initialize file system
initialize database
while application is running:
if new user session:
create new session folder in file system
prompt user for input via web interface
user_input = get user input from web interface
idea = GPT_Actor_Idea_Generator.generate(user_input)
expanded_content = GPT_Actor_Content_Expander.expand(idea)
refined_content = GPT_Actor_Refinement_Agent.refine(expanded_content)
validation_result = GPT_Actor_Validator.validate(refined_content)
if validation_result is not satisfactory:
prompt user for feedback
continue refining process based on feedback
visual_content = GPT_Actor_Visualizer.visualize(refined_content)
summary = GPT_Actor_Summarizer.summarize(refined_content)
save outputs to session folder
display links to outputs on web interface
if user ends session:
archive session data in database
close session
The architecture described here provides a high-level view of how the system components interact and how the GPT actors contribute to the creative process. Each actor's purpose and description are outlined, and the pseudocode provides a guide for the overall logic flow. The data structures facilitate the passing and handling of information between the system components and the user.
{
"name" : "Agent Flow Design",
"description" : "Break down the text into a data structure.",
"mainInput" : {
"name" : "Concept",
"description" : "A data structure containing the idea, keywords, and any initial thoughts or expansions.",
"type" : "Data Structure"
},
"logicFlow" : {
"items" : [ {
"name" : "Idea Generation",
"description" : "To generate initial creative concepts based on user input or predefined prompts.",
"actors" : [ "Idea Generator (Actor 1)" ],
"inputs" : [ ],
"output" : {
"name" : "Concept",
"description" : "A data structure containing the idea, keywords, and any initial thoughts or expansions.",
"type" : "Data Structure"
}
}, {
"name" : "Concept Refinement",
"description" : "To refine and expand upon the initial ideas generated by Actor 1.",
"actors" : [ "Concept Refiner (Actor 2)" ],
"inputs" : [ {
"name" : "Concept",
"description" : "A data structure containing the idea, keywords, and any initial thoughts or expansions.",
"type" : "Data Structure"
} ],
"output" : {
"name" : "Concept",
"description" : "A data structure containing the idea, keywords, and any refined thoughts or expansions.",
"type" : "Data Structure"
}
}, {
"name" : "Structure Creation",
"description" : "To create a structured outline or framework for the creative output.",
"actors" : [ "Structure Creator (Actor 3)" ],
"inputs" : [ {
"name" : "Concept",
"description" : "A data structure containing the idea, keywords, and any refined thoughts or expansions.",
"type" : "Data Structure"
} ],
"output" : {
"name" : "Structure",
"description" : "A data structure representing the organized framework or outline of the creative work.",
"type" : "Data Structure"
}
}, {
"name" : "Content Production",
"description" : "To generate the main content or body of the creative work.",
"actors" : [ "Content Producer (Actor 4)" ],
"inputs" : [ {
"name" : "Structure",
"description" : "A data structure representing the organized framework or outline of the creative work.",
"type" : "Data Structure"
} ],
"output" : {
"name" : "Content",
"description" : "A data structure containing the main body of the creative work, including text, images, or other media.",
"type" : "Data Structure"
}
}, {
"name" : "Editing and Polishing",
"description" : "To review and polish the content produced by Actor 4.",
"actors" : [ "Editor and Polisher (Actor 5)" ],
"inputs" : [ {
"name" : "Content",
"description" : "A data structure containing the main body of the creative work, including text, images, or other media.",
"type" : "Data Structure"
} ],
"output" : {
"name" : "Content",
"description" : "A data structure containing the main body of the creative work, including text, images, or other media.",
"type" : "Data Structure"
}
}, {
"name" : "Final Review",
"description" : "To provide a final review and prepare the creative work for presentation.",
"actors" : [ "Reviewer (Actor 6)" ],
"inputs" : [ {
"name" : "Content",
"description" : "A data structure containing the main body of the creative work, including text, images, or other media.",
"type" : "Data Structure"
} ],
"output" : {
"name" : "FinalOutput",
"description" : "A data structure representing the polished and finalized creative work ready for presentation.",
"type" : "Data Structure"
}
} ]
}
}
To expand on the design provided, let's detail the requirements for each GPT actor within the system:
public class ConceptData {
private List<String> concepts;
private List<String> facts;
private List<Relationship> relationships;
// getters and setters
}
public class QuestionStems {
private List<QuestionStem> stems;
// getters and setters
}
public class QuestionsWithDistractors {
private List<Question> questions;
// getters and setters
}
public class ValidatedQuestions {
private List<Question> validatedQuestions;
// getters and setters
}
public class AdjustedQuestions {
private List<Question> adjustedQuestions;
// getters and setters
}
Each actor will be responsible for a specific part of the question generation process, and their outputs will be structured to facilitate the flow of data through the system. The use of parsed actors ensures that the output can be easily integrated into the application logic, allowing for a seamless transition from one step to the next in the question generation pipeline.
{
"actors" : [ {
"name" : "ContentAnalyzerActor",
"description" : "To analyze the uploaded material and extract key concepts, facts, and relationships that will serve as the basis for generating questions.",
"type" : "parsed",
"resultClass" : "ContentAnalysisResult",
"simpleClassName" : "ContentAnalysisResult"
}, {
"name" : "QuestionGeneratorActor",
"description" : "To generate the stems of multiple-choice questions based on the extracted content.",
"type" : "parsed",
"resultClass" : "QuestionStems",
"simpleClassName" : "QuestionStems"
}, {
"name" : "DistractorGeneratorActor",
"description" : "To create plausible incorrect answer choices (distractors) for each question.",
"type" : "parsed",
"resultClass" : "Distractors",
"simpleClassName" : "Distractors"
}, {
"name" : "ValidatorActor",
"description" : "To validate the correctness of the generated questions and their answers.",
"type" : "parsed",
"resultClass" : "ValidationResult",
"simpleClassName" : "ValidationResult"
}, {
"name" : "DifficultyAdjusterActor",
"description" : "To adjust the complexity of questions based on the specified depth of knowledge.",
"type" : "parsed",
"resultClass" : "AdjustedQuestions",
"simpleClassName" : "AdjustedQuestions"
} ]
}
import com.simiacryptus.jopenai.describe.Description
import com.simiacryptus.jopenai.models.ChatModels
import com.simiacryptus.jopenai.proxy.ValidatedObject
import com.simiacryptus.skyenet.core.actors.ParsedActor
import java.util.function.Function
data class Distractors(
@Description("A map of question stems to their respective list of distractors.")
val distractorMap: Map<String, List<String>>
) : ValidatedObject {
override fun validate() = when {
distractorMap.isEmpty() -> "distractorMap cannot be empty"
distractorMap.any { it.value.isEmpty() } -> "Each question stem must have at least one distractor"
else -> null
}
}
interface DistractorParser : Function<String, Distractors> {
@Description("Parse the text response into a map of question stems to their respective list of distractors.")
override fun apply(text: String): Distractors
}
val distractorGeneratorActor = ParsedActor<Distractors>(
parserClass = DistractorParser::class.java,
model = ChatModels.GPT35Turbo,
prompt = """
You are an assistant that generates plausible distractors for multiple-choice questions.
Given a question stem, produce a list of incorrect answers that are thematically consistent but clearly distinct from the correct answer.
Here is the question stem: [Question Stem]
Please provide a list of distractors.
""".trimIndent()
)
import com.simiacryptus.jopenai.models.ChatModels
import com.simiacryptus.jopenai.proxy.ValidatedObject
import com.simiacryptus.skyenet.core.actors.ParsedActor
import java.util.function.Function
// Define the data class for the adjusted questions
data class AdjustedQuestion(
val stem: String,
val options: List<String>,
val correct: Int,
val difficulty: String
) : ValidatedObject {
override fun validate() = when {
stem.isBlank() -> "Question stem is required"
options.isEmpty() -> "Options are required"
correct !in options.indices -> "Correct answer index is out of bounds"
difficulty.isBlank() -> "Difficulty level is required"
else -> null
}
}
data class AdjustedQuestions(
val questions: List<AdjustedQuestion>
) : ValidatedObject {
override fun validate() = when {
questions.any { it.validate() != null } -> "All questions must be valid"
else -> null
}
}
// Define the parser interface
interface AdjustedQuestionsParser : Function<String, AdjustedQuestions> {
override fun apply(text: String): AdjustedQuestions
}
// Instantiate the difficultyAdjusterActor
val difficultyAdjusterActor = ParsedActor<AdjustedQuestions>(
parserClass = AdjustedQuestionsParser::class.java,
model = ChatModels.GPT35Turbo,
prompt = """
You are an assistant that adjusts the difficulty level of multiple-choice questions.
Given a list of questions and a specified difficulty level, modify the questions to match the desired difficulty.
Output the adjusted questions in a structured format.
""".trimIndent()
)
import com.simiacryptus.jopenai.describe.Description
import com.simiacryptus.jopenai.models.ChatModels
import com.simiacryptus.jopenai.proxy.ValidatedObject
import com.simiacryptus.skyenet.core.actors.ParsedActor
import java.util.function.Function
// Define the data class that will hold the validation result
data class ValidationResult(
@Description("The status indicating if the question is valid.")
val isValid: Boolean,
@Description("The feedback message explaining why the question is valid or not.")
val feedback: String? = null
) : ValidatedObject {
override fun validate() = when {
feedback.isNullOrBlank() -> "Feedback message is required"
else -> null
}
}
// Define the parser interface
interface ValidatorParser : Function<String, ValidationResult> {
@Description("Parse the text response into a ValidationResult.")
override fun apply(text: String): ValidationResult
}
// Instantiate the ParsedActor with the ValidatorParser
val validatorActor = ParsedActor<ValidationResult>(
parserClass = ValidatorParser::class.java,
model = ChatModels.GPT35Turbo,
prompt = """
You are an assistant that validates the correctness of multiple-choice questions.
Given a question with its correct answer and distractors, determine if the question is valid.
Provide feedback on the validity of the question.
""".trimIndent()
)
import com.simiacryptus.jopenai.describe.Description
import com.simiacryptus.jopenai.models.ChatModels
import com.simiacryptus.jopenai.proxy.ValidatedObject
import com.simiacryptus.skyenet.core.actors.ParsedActor
import java.util.function.Function
interface ContentAnalysisParser : Function<String, ContentAnalysisResult> {
@Description("Parse the text into a structured content analysis result.")
override fun apply(text: String): ContentAnalysisResult
}
data class ContentAnalysisResult(
@Description("The key concepts identified in the content.")
val concepts: List<String>? = null,
@Description("The key facts identified in the content.")
val facts: List<String>? = null,
@Description("The relationships identified between concepts in the content.")
val relationships: List<String>? = null
) : ValidatedObject {
override fun validate() = when {
concepts.isNullOrEmpty() -> "At least one concept is required"
facts.isNullOrEmpty() -> "At least one fact is required"
relationships.isNullOrEmpty() -> "At least one relationship is required"
else -> null
}
}
val contentAnalyzerActor = ParsedActor<ContentAnalysisResult>(
parserClass = ContentAnalysisParser::class.java,
model = ChatModels.GPT35Turbo,
prompt = """
You are an assistant that analyzes educational content. Your task is to identify and list the key concepts, facts, and relationships present in the content provided.
Content:
""".trimIndent()
)
import com.simiacryptus.jopenai.describe.Description
import com.simiacryptus.jopenai.models.ChatModels
import com.simiacryptus.jopenai.proxy.ValidatedObject
import com.simiacryptus.skyenet.core.actors.ParsedActor
import java.util.function.Function
data class QuestionStems(
@Description("A list of generated question stems.")
val stems: List<String>
) : ValidatedObject {
override fun validate() = when {
stems.isEmpty() -> "At least one question stem is required"
stems.any { it.isBlank() } -> "Question stems cannot be blank"
else -> null
}
}
interface QuestionStemsParser : Function<String, QuestionStems> {
@Description("Parse the text response into a list of question stems.")
override fun apply(text: String): QuestionStems
}
val questionGeneratorActor = ParsedActor<QuestionStems>(
parserClass = QuestionStemsParser::class.java,
model = ChatModels.GPT35Turbo,
prompt = """
You are an assistant that generates multiple-choice question stems. Given a topic or material, produce a list of question stems that are clear, concise, and directly related to the topic. Ensure that the questions are open-ended and suitable for multiple-choice answers.
Topic: "The life cycle of a butterfly"
Generate question stems:
""".trimIndent()
)
fun ideaGeneration() {
val task = ui.newTask()
try {
task.header("Idea Generation for Multiple Choice Questions")
// Step 1: Get the topic from the user
task.add("Please enter the topic or material for which you want to generate multiple-choice questions:")
val topic = ui.textInput { topic ->
task.add("Received topic: $topic")
// Step 2: Analyze the content to extract concepts, facts, and relationships
task.add("Analyzing the content...")
val contentAnalysisResult = contentAnalyzerActor.answer(listOf(topic), api = api)
task.add("Content analysis complete. Extracted concepts and facts.")
// Step 3: Generate question stems based on the analyzed content
task.add("Generating question stems...")
val questionStems = questionGeneratorActor.answer(listOf(topic), api = api)
task.add("Question stems generated.")
// Step 4: Generate distractors for each question stem
task.add("Generating distractors for each question stem...")
val distractors = questionStems.obj.stems.map { stem ->
stem to distractorGeneratorActor.answer(listOf(stem), api = api).obj.distractorMap[stem]
}.toMap()
task.add("Distractors generated for all question stems.")
// Step 5: Validate the generated questions and distractors
task.add("Validating questions...")
val validatedQuestions = distractors.map { (stem, distractors) ->
validatorActor.answer(listOf(stem, distractors.toString()), api = api).obj
}
task.add("Questions validated.")
// Step 6: Adjust the difficulty level of the questions (if needed)
// This step would involve interaction with the user to set the desired difficulty level
// and then using the difficultyAdjusterActor to adjust the questions accordingly.
// Step 7: Compile the questions into a database and present to the user
// This step would involve storing the validated and adjusted questions into a database
// and providing the user with a link to download or interact with the database.
task.complete("Multiple-choice question generation is complete. You can now review and download the questions.")
}
} catch (e: Throwable) {
task.error(e)
throw e
}
}
data class Concept(
val description: String,
val relatedConcepts: List<String>,
val additionalInfo: String?
)
interface ConceptRefinementParser : Function<String, Concept> {
@Description("Parse the text into a refined concept.")
override fun apply(text: String): Concept
}
val conceptRefinementActor = ParsedActor<Concept>(
parserClass = ConceptRefinementParser::class.java,
model = ChatModels.GPT35Turbo,
prompt = """
You are an assistant that refines concepts to make them more comprehensive and detailed.
Take the following concept and enhance it by providing a more in-depth description,
suggesting additional related concepts, and any other relevant information that could
be useful for understanding the concept better.
Current Concept Description: ${'$'}description
Related Concepts: ${'$'}relatedConcepts
Additional Information: ${'$'}additionalInfo
Please refine the concept:
""".trimIndent()
)
fun conceptRefinement(concept: Concept) {
val task = ui.newTask()
try {
task.header("Concept Refinement Process")
// Display the current concept to the user
task.add("Current concept: ${concept.description}")
task.verbose("Related concepts: ${concept.relatedConcepts.joinToString(", ")}")
task.verbose("Additional information: ${concept.additionalInfo ?: "None provided"}")
// Refine the concept using the GPT actor
task.add("Refining the concept...")
val prompt = conceptRefinementActor.prompt.replace(
"${'$'}description", concept.description
).replace(
"${'$'}relatedConcepts", concept.relatedConcepts.joinToString(", ")
).replace(
"${'$'}additionalInfo", concept.additionalInfo ?: "None provided"
)
val refinedConcept = conceptRefinementActor.answer(listOf(prompt), api = api).obj
task.add("Concept refinement complete.")
// Display the refined concept to the user
task.add("Refined concept: ${refinedConcept.description}")
task.verbose("Related concepts: ${refinedConcept.relatedConcepts.joinToString(", ")}")
task.verbose("Additional information: ${refinedConcept.additionalInfo ?: "None provided"}")
task.complete("Concept refinement is complete. Review the refined concept above.")
} catch (e: Throwable) {
task.error(e)
throw e
}
}
// Define the parser interface for concept structure creation
interface ConceptStructureCreationParser : Function<String, Concept> {
@Description("Parse the text into a structured concept.")
override fun apply(text: String): Concept
}
// Define the actor for concept structure creation
val conceptStructureCreationActor = ParsedActor<Concept>(
parserClass = ConceptStructureCreationParser::class.java,
model = ChatModels.GPT35Turbo,
prompt = """
You are an assistant that creates structured concepts from a given description, related concepts, and additional information.
Generate a structured concept with clear definitions, examples, and applications.
Concept Description: ${'$'}description
Related Concepts: ${'$'}relatedConcepts
Additional Information: ${'$'}additionalInfo
Please create a structured concept:
""".trimIndent()
)
// Implement the structureCreation function
fun structureCreation(concept: Concept) {
val task = ui.newTask()
try {
task.header("Concept Structure Creation")
// Display the current concept to the user
task.add("Current concept: ${concept.description}")
task.verbose("Related concepts: ${concept.relatedConcepts.joinToString(", ")}")
task.verbose("Additional information: ${concept.additionalInfo ?: "None provided"}")
// Create the structured concept using the GPT actor
task.add("Creating the structured concept...")
val prompt = conceptStructureCreationActor.prompt.replace(
"${'$'}description", concept.description
).replace(
"${'$'}relatedConcepts", concept.relatedConcepts.joinToString(", ")
).replace(
"${'$'}additionalInfo", concept.additionalInfo ?: "None provided"
)
val structuredConcept = conceptStructureCreationActor.answer(listOf(prompt), api = api).obj
task.add("Concept structure creation complete.")
// Display the structured concept to the user
task.add("Structured concept: ${structuredConcept.description}")
task.verbose("Related concepts: ${structuredConcept.relatedConcepts.joinToString(", ")}")
task.verbose("Additional information: ${structuredConcept.additionalInfo ?: "None provided"}")
task.complete("Concept structure creation is complete. Review the structured concept above.")
} catch (e: Throwable) {
task.error(e)
throw e
}
}
data class DataStructure(
val topic: String,
val keyConcepts: List<String>,
val additionalInfo: String? = null
)
fun contentProduction(structure: DataStructure) {
val task = ui.newTask()
try {
task.header("Content Production for Multiple Choice Questions")
// Step 1: Analyze the content to extract concepts, facts, and relationships
task.add("Analyzing the content for the topic: ${structure.topic}")
val contentAnalysisResult = contentAnalyzerActor.answer(listOf(structure.topic), api = api)
task.add("Content analysis complete. Extracted concepts and facts.")
// Step 2: Generate question stems based on the analyzed content
task.add("Generating question stems...")
val questionStems = questionGeneratorActor.answer(listOf(structure.topic), api = api)
task.add("Question stems generated.")
// Step 3: Generate distractors for each question stem
task.add("Generating distractors for each question stem...")
val distractors = questionStems.obj.stems.map { stem ->
stem to distractorGeneratorActor.answer(listOf(stem), api = api).obj.distractorMap[stem]
}.toMap()
task.add("Distractors generated for all question stems.")
// Step 4: Validate the generated questions and distractors
task.add("Validating questions...")
val validatedQuestions = distractors.map { (stem, distractors) ->
validatorActor.answer(listOf(stem, distractors.toString()), api = api).obj
}
task.add("Questions validated.")
// Step 5: Adjust the difficulty level of the questions (if needed)
// This step would involve interaction with the user to set the desired difficulty level
// and then using the difficultyAdjusterActor to adjust the questions accordingly.
// Step 6: Compile the questions into a database and present to the user
// This step would involve storing the validated and adjusted questions into a database
// and providing the user with a link to download or interact with the database.
task.complete("Content production for multiple-choice questions is complete. You can now review and download the questions.")
} catch (e: Throwable) {
task.error(e)
throw e
}
}
fun editingAndPolishing(content: DataStructure) {
val task = ui.newTask()
try {
task.header("Editing and Polishing Content")
// Step 1: Analyze the content to extract concepts, facts, and relationships
task.add("Analyzing the content for the topic: ${content.topic}")
val contentAnalysisResult = contentAnalyzerActor.answer(listOf(content.topic), api = api).obj
task.add("Content analysis complete. Extracted concepts and facts: ${contentAnalysisResult.concepts}")
// Step 2: Generate question stems based on the analyzed content
task.add("Generating question stems...")
val questionStemsResult = questionGeneratorActor.answer(listOf(content.topic), api = api).obj
task.add("Question stems generated: ${questionStemsResult.stems}")
// Step 3: Generate distractors for each question stem
task.add("Generating distractors for each question stem...")
val distractorsResult = questionStemsResult.stems.map { stem ->
stem to distractorGeneratorActor.answer(listOf(stem), api = api).obj.distractorMap[stem]
}.toMap()
task.add("Distractors generated for all question stems.")
// Step 4: Validate the generated questions and distractors
task.add("Validating questions...")
val validatedQuestions = distractorsResult.map { (stem, distractors) ->
validatorActor.answer(listOf(stem, distractors.toString()), api = api).obj
}
task.add("Questions validated.")
// Step 5: Adjust the difficulty level of the questions (if needed)
// This step would involve interaction with the user to set the desired difficulty level
// and then using the difficultyAdjusterActor to adjust the questions accordingly.
// Step 6: Compile the questions into a database and present to the user
// This step would involve storing the validated and adjusted questions into a database
// and providing the user with a link to download or interact with the database.
task.complete("Editing and polishing of content is complete. You can now review and download the questions.")
} catch (e: Throwable) {
task.error(e)
throw e
}
}
fun finalReview(content: DataStructure) {
val task = ui.newTask()
try {
task.header("Final Review of Generated Content")
// Step 1: Analyze the content to extract concepts, facts, and relationships
task.add("Analyzing the content for the topic: ${content.topic}")
val contentAnalysisResult = contentAnalyzerActor.answer(listOf(content.topic), api = api).obj
task.add("Content analysis complete. Extracted concepts and facts: ${contentAnalysisResult.concepts}")
// Step 2: Generate question stems based on the analyzed content
task.add("Generating question stems...")
val questionStemsResult = questionGeneratorActor.answer(listOf(content.topic), api = api).obj
task.add("Question stems generated: ${questionStemsResult.stems}")
// Step 3: Generate distractors for each question stem
task.add("Generating distractors for each question stem...")
val distractorsResult = questionStemsResult.stems.map { stem ->
stem to distractorGeneratorActor.answer(listOf(stem), api = api).obj.distractorMap[stem]
}.toMap()
task.add("Distractors generated for all question stems.")
// Step 4: Validate the generated questions and distractors
task.add("Validating questions...")
val validatedQuestions = distractorsResult.map { (stem, distractors) ->
validatorActor.answer(listOf(stem, distractors.toString()), api = api).obj
}
task.add("Questions validated.")
// Step 5: Adjust the difficulty level of the questions (if needed)
// This step would involve interaction with the user to set the desired difficulty level
// and then using the difficultyAdjusterActor to adjust the questions accordingly.
// Step 6: Compile the questions into a database and present to the user
// This step would involve storing the validated and adjusted questions into a database
// and providing the user with a link to download or interact with the database.
task.complete("Final review of content is complete. You can now review and download the questions.")
} catch (e: Throwable) {
task.error(e)
throw e
}
}
fun agentFlowDesign(concept: DataStructure) {
val task = ui.newTask()
try {
task.header("Agent Flow Design for Test Generator")
// Step 1: Analyze the content to extract concepts, facts, and relationships
task.add("Analyzing the content for the topic: ${concept.topic}")
val contentAnalysisResult = contentAnalyzerActor.answer(listOf(concept.topic), api = api).obj
task.add("Content analysis complete. Extracted concepts and facts: ${contentAnalysisResult.concepts}")
// Step 2: Generate question stems based on the analyzed content
task.add("Generating question stems...")
val questionStemsResult = questionGeneratorActor.answer(listOf(concept.topic), api = api).obj
task.add("Question stems generated: ${questionStemsResult.stems}")
// Step 3: Generate distractors for each question stem
task.add("Generating distractors for each question stem...")
val distractorsResult = questionStemsResult.stems.map { stem ->
stem to distractorGeneratorActor.answer(listOf(stem), api = api).obj.distractorMap[stem]
}.toMap()
task.add("Distractors generated for all question stems.")
// Step 4: Validate the generated questions and distractors
task.add("Validating questions...")
val validatedQuestions = distractorsResult.map { (stem, distractors) ->
validatorActor.answer(listOf(stem, distractors.toString()), api = api).obj
}
task.add("Questions validated.")
// Step 5: Adjust the difficulty level of the questions (if needed)
// This step would involve interaction with the user to set the desired difficulty level
// and then using the difficultyAdjusterActor to adjust the questions accordingly.
// Step 6: Compile the questions into a database and present to the user
// This step would involve storing the validated and adjusted questions into a database
// and providing the user with a link to download or interact with the database.
task.complete("Agent flow design for test generator is complete. You can now review and download the questions.")
} catch (e: Throwable) {
task.error(e)
throw e
}
}
import com.simiacryptus.jopenai.API
import com.simiacryptus.jopenai.describe.Description
import com.simiacryptus.jopenai.models.ChatModels
import com.simiacryptus.jopenai.proxy.ValidatedObject
import com.simiacryptus.skyenet.core.actors.ActorSystem
import com.simiacryptus.skyenet.core.actors.BaseActor
import com.simiacryptus.skyenet.core.actors.CodingActor
import com.simiacryptus.skyenet.core.actors.ImageActor
import com.simiacryptus.skyenet.core.actors.ParsedActor
import com.simiacryptus.skyenet.core.platform.Session
import com.simiacryptus.skyenet.core.platform.StorageInterface
import com.simiacryptus.skyenet.core.platform.User
import com.simiacryptus.skyenet.core.platform.file.DataStorage
import com.simiacryptus.skyenet.webui.application.ApplicationInterface
import com.simiacryptus.skyenet.webui.application.ApplicationServer
import com.simiacryptus.skyenet.webui.session.*
import java.awt.image.BufferedImage
import java.util.function.Function
import org.slf4j.LoggerFactory
open class AgentFlowDesignApp(
applicationName: String = "Agent Flow Design",
) : ApplicationServer(
applicationName = applicationName,
) {
data class Settings(
val model: ChatModels = ChatModels.GPT35Turbo,
val temperature: Double = 0.1,
)
override val settingsClass: Class<*> get() = Settings::class.java
@Suppress("UNCHECKED_CAST") override fun <T:Any> initSettings(session: Session): T? = Settings() as T
override fun userMessage(
session: Session,
user: User?,
userMessage: String,
ui: ApplicationInterface,
api: API
) {
try {
val settings = getSettings<Settings>(session, user)
AgentFlowDesignAgent(
user = user,
session = session,
dataStorage = dataStorage,
api = api,
ui = ui,
model = settings?.model ?: ChatModels.GPT35Turbo,
temperature = settings?.temperature ?: 0.3,
).agentFlowDesign(userMessage)
} catch (e: Throwable) {
log.warn("Error", e)
}
}
companion object {
private val log = LoggerFactory.getLogger(AgentFlowDesignApp::class.java)
}
}
open class AgentFlowDesignAgent(
user: User?,
session: Session,
dataStorage: StorageInterface,
val ui: ApplicationInterface,
val api: API,
model: ChatModels = ChatModels.GPT35Turbo,
temperature: Double = 0.3,
) : ActorSystem<AgentFlowDesignActors.ActorType>(AgentFlowDesignActors(
model = model,
temperature = temperature,
).actorMap, dataStorage, user, session) {
@Suppress("UNCHECKED_CAST")
private val contentAnalyzerActor by lazy { getActor(AgentFlowDesignActors.ActorType.CONTENT_ANALYZER_ACTOR) as ParsedActor<AgentFlowDesignActors.ContentAnalysisResult> }
private val questionGeneratorActor by lazy { getActor(AgentFlowDesignActors.ActorType.QUESTION_GENERATOR_ACTOR) as ParsedActor<AgentFlowDesignActors.QuestionStems> }
private val distractorGeneratorActor by lazy { getActor(AgentFlowDesignActors.ActorType.DISTRACTOR_GENERATOR_ACTOR) as ParsedActor<AgentFlowDesignActors.Distractors> }
private val validatorActor by lazy { getActor(AgentFlowDesignActors.ActorType.VALIDATOR_ACTOR) as ParsedActor<AgentFlowDesignActors.ValidationResult> }
private val difficultyAdjusterActor by lazy { getActor(AgentFlowDesignActors.ActorType.DIFFICULTY_ADJUSTER_ACTOR) as ParsedActor<AgentFlowDesignActors.AdjustedQuestions> }
fun agentFlowDesign(concept: DataStructure) {
val task = ui.newTask()
try {
task.header("Agent Flow Design for Test Generator")
// Step 1: Analyze the content to extract concepts, facts, and relationships
task.add("Analyzing the content for the topic: ${concept.topic}")
val contentAnalysisResult = contentAnalyzerActor.answer(listOf(concept.topic), api = api).obj
task.add("Content analysis complete. Extracted concepts and facts: ${contentAnalysisResult.concepts}")
// Step 2: Generate question stems based on the analyzed content
task.add("Generating question stems...")
val questionStemsResult = questionGeneratorActor.answer(listOf(concept.topic), api = api).obj
task.add("Question stems generated: ${questionStemsResult.stems}")
// Step 3: Generate distractors for each question stem
task.add("Generating distractors for each question stem...")
val distractorsResult = questionStemsResult.stems.map { stem ->
stem to distractorGeneratorActor.answer(listOf(stem), api = api).obj.distractorMap[stem]
}.toMap()
task.add("AgentFlowDesignActors.Distractors generated for all question stems.")
// Step 4: Validate the generated questions and distractors
task.add("Validating questions...")
val validatedQuestions = distractorsResult.map { (stem, distractors) ->
validatorActor.answer(listOf(stem, distractors.toString()), api = api).obj
}
task.add("Questions validated.")
// Step 5: Adjust the difficulty level of the questions (if needed)
// This step would involve interaction with the user to set the desired difficulty level
// and then using the difficultyAdjusterActor to adjust the questions accordingly.
// Step 6: Compile the questions into a database and present to the user
// This step would involve storing the validated and adjusted questions into a database
// and providing the user with a link to download or interact with the database.
task.complete("Agent flow design for test generator is complete. You can now review and download the questions.")
} catch (e: Throwable) {
task.error(e)
throw e
}
}
data class DataStructure(
val topic: String,
val keyConcepts: List<String>,
val additionalInfo: String? = null
)
fun contentProduction(structure: DataStructure) {
val task = ui.newTask()
try {
task.header("Content Production for Multiple Choice Questions")
// Step 1: Analyze the content to extract concepts, facts, and relationships
task.add("Analyzing the content for the topic: ${structure.topic}")
val contentAnalysisResult = contentAnalyzerActor.answer(listOf(structure.topic), api = api)
task.add("Content analysis complete. Extracted concepts and facts.")
// Step 2: Generate question stems based on the analyzed content
task.add("Generating question stems...")
val questionStems = questionGeneratorActor.answer(listOf(structure.topic), api = api)
task.add("Question stems generated.")
// Step 3: Generate distractors for each question stem
task.add("Generating distractors for each question stem...")
val distractors = questionStems.obj.stems.map { stem ->
stem to distractorGeneratorActor.answer(listOf(stem), api = api).obj.distractorMap[stem]
}.toMap()
task.add("AgentFlowDesignActors.Distractors generated for all question stems.")
// Step 4: Validate the generated questions and distractors
task.add("Validating questions...")
val validatedQuestions = distractors.map { (stem, distractors) ->
validatorActor.answer(listOf(stem, distractors.toString()), api = api).obj
}
task.add("Questions validated.")
// Step 5: Adjust the difficulty level of the questions (if needed)
// This step would involve interaction with the user to set the desired difficulty level
// and then using the difficultyAdjusterActor to adjust the questions accordingly.
// Step 6: Compile the questions into a database and present to the user
// This step would involve storing the validated and adjusted questions into a database
// and providing the user with a link to download or interact with the database.
task.complete("Content production for multiple-choice questions is complete. You can now review and download the questions.")
} catch (e: Throwable) {
task.error(e)
throw e
}
}
fun finalReview(content: DataStructure) {
val task = ui.newTask()
try {
task.header("Final Review of Generated Content")
// Step 1: Analyze the content to extract concepts, facts, and relationships
task.add("Analyzing the content for the topic: ${content.topic}")
val contentAnalysisResult = contentAnalyzerActor.answer(listOf(content.topic), api = api).obj
task.add("Content analysis complete. Extracted concepts and facts: ${contentAnalysisResult.concepts}")
// Step 2: Generate question stems based on the analyzed content
task.add("Generating question stems...")
val questionStemsResult = questionGeneratorActor.answer(listOf(content.topic), api = api).obj
task.add("Question stems generated: ${questionStemsResult.stems}")
// Step 3: Generate distractors for each question stem
task.add("Generating distractors for each question stem...")
val distractorsResult = questionStemsResult.stems.map { stem ->
stem to distractorGeneratorActor.answer(listOf(stem), api = api).obj.distractorMap[stem]
}.toMap()
task.add("AgentFlowDesignActors.Distractors generated for all question stems.")
// Step 4: Validate the generated questions and distractors
task.add("Validating questions...")
val validatedQuestions = distractorsResult.map { (stem, distractors) ->
validatorActor.answer(listOf(stem, distractors.toString()), api = api).obj
}
task.add("Questions validated.")
// Step 5: Adjust the difficulty level of the questions (if needed)
// This step would involve interaction with the user to set the desired difficulty level
// and then using the difficultyAdjusterActor to adjust the questions accordingly.
// Step 6: Compile the questions into a database and present to the user
// This step would involve storing the validated and adjusted questions into a database
// and providing the user with a link to download or interact with the database.
task.complete("Final review of content is complete. You can now review and download the questions.")
} catch (e: Throwable) {
task.error(e)
throw e
}
}
fun ideaGeneration() {
val task = ui.newTask()
try {
task.header("Idea Generation for Multiple Choice Questions")
// Step 1: Get the topic from the user
task.add("Please enter the topic or material for which you want to generate multiple-choice questions:")
val topic = ui.textInput { topic ->
task.add("Received topic: $topic")
// Step 2: Analyze the content to extract concepts, facts, and relationships
task.add("Analyzing the content...")
val contentAnalysisResult = contentAnalyzerActor.answer(listOf(topic), api = api)
task.add("Content analysis complete. Extracted concepts and facts.")
// Step 3: Generate question stems based on the analyzed content
task.add("Generating question stems...")
val questionStems = questionGeneratorActor.answer(listOf(topic), api = api)
task.add("Question stems generated.")
// Step 4: Generate distractors for each question stem
task.add("Generating distractors for each question stem...")
val distractors = questionStems.obj.stems.map { stem ->
stem to distractorGeneratorActor.answer(listOf(stem), api = api).obj.distractorMap[stem]
}.toMap()
task.add("AgentFlowDesignActors.Distractors generated for all question stems.")
// Step 5: Validate the generated questions and distractors
task.add("Validating questions...")
val validatedQuestions = distractors.map { (stem, distractors) ->
validatorActor.answer(listOf(stem, distractors.toString()), api = api).obj
}
task.add("Questions validated.")
// Step 6: Adjust the difficulty level of the questions (if needed)
// This step would involve interaction with the user to set the desired difficulty level
// and then using the difficultyAdjusterActor to adjust the questions accordingly.
// Step 7: Compile the questions into a database and present to the user
// This step would involve storing the validated and adjusted questions into a database
// and providing the user with a link to download or interact with the database.
task.complete("Multiple-choice question generation is complete. You can now review and download the questions.")
}
} catch (e: Throwable) {
task.error(e)
throw e
}
}
data class Concept(
val description: String,
val relatedConcepts: List<String>,
val additionalInfo: String?
)
interface ConceptRefinementParser : Function<String, Concept> {
@Description("Parse the text into a refined concept.")
override fun apply(text: String): Concept
}
val conceptRefinementActor = ParsedActor<Concept>(
parserClass = ConceptRefinementParser::class.java,
model = ChatModels.GPT35Turbo,
prompt = """
You are an assistant that refines concepts to make them more comprehensive and detailed.
Take the following concept and enhance it by providing a more in-depth description,
suggesting additional related concepts, and any other relevant information that could
be useful for understanding the concept better.
Current Concept Description: ${'$'}description
Related Concepts: ${'$'}relatedConcepts
Additional Information: ${'$'}additionalInfo
Please refine the concept:
""".trimIndent()
)
fun conceptRefinement(concept: Concept) {
val task = ui.newTask()
try {
task.header("Concept Refinement Process")
// Display the current concept to the user
task.add("Current concept: ${concept.description}")
task.verbose("Related concepts: ${concept.relatedConcepts.joinToString(", ")}")
task.verbose("Additional information: ${concept.additionalInfo ?: "None provided"}")
// Refine the concept using the GPT actor
task.add("Refining the concept...")
val prompt = conceptRefinementActor.prompt.replace(
"${'$'}description", concept.description
).replace(
"${'$'}relatedConcepts", concept.relatedConcepts.joinToString(", ")
).replace(
"${'$'}additionalInfo", concept.additionalInfo ?: "None provided"
)
val refinedConcept = conceptRefinementActor.answer(listOf(prompt), api = api).obj
task.add("Concept refinement complete.")
// Display the refined concept to the user
task.add("Refined concept: ${refinedConcept.description}")
task.verbose("Related concepts: ${refinedConcept.relatedConcepts.joinToString(", ")}")
task.verbose("Additional information: ${refinedConcept.additionalInfo ?: "None provided"}")
task.complete("Concept refinement is complete. Review the refined concept above.")
} catch (e: Throwable) {
task.error(e)
throw e
}
}
// Define the parser interface for concept structure creation
interface ConceptStructureCreationParser : Function<String, Concept> {
@Description("Parse the text into a structured concept.")
override fun apply(text: String): Concept
}
// Define the actor for concept structure creation
val conceptStructureCreationActor = ParsedActor<Concept>(
parserClass = ConceptStructureCreationParser::class.java,
model = ChatModels.GPT35Turbo,
prompt = """
You are an assistant that creates structured concepts from a given description, related concepts, and additional information.
Generate a structured concept with clear definitions, examples, and applications.
Concept Description: ${'$'}description
Related Concepts: ${'$'}relatedConcepts
Additional Information: ${'$'}additionalInfo
Please create a structured concept:
""".trimIndent()
)
// Implement the structureCreation function
fun structureCreation(concept: Concept) {
val task = ui.newTask()
try {
task.header("Concept Structure Creation")
// Display the current concept to the user
task.add("Current concept: ${concept.description}")
task.verbose("Related concepts: ${concept.relatedConcepts.joinToString(", ")}")
task.verbose("Additional information: ${concept.additionalInfo ?: "None provided"}")
// Create the structured concept using the GPT actor
task.add("Creating the structured concept...")
val prompt = conceptStructureCreationActor.prompt.replace(
"${'$'}description", concept.description
).replace(
"${'$'}relatedConcepts", concept.relatedConcepts.joinToString(", ")
).replace(
"${'$'}additionalInfo", concept.additionalInfo ?: "None provided"
)
val structuredConcept = conceptStructureCreationActor.answer(listOf(prompt), api = api).obj
task.add("Concept structure creation complete.")
// Display the structured concept to the user
task.add("Structured concept: ${structuredConcept.description}")
task.verbose("Related concepts: ${structuredConcept.relatedConcepts.joinToString(", ")}")
task.verbose("Additional information: ${structuredConcept.additionalInfo ?: "None provided"}")
task.complete("Concept structure creation is complete. Review the structured concept above.")
} catch (e: Throwable) {
task.error(e)
throw e
}
}
fun editingAndPolishing(content: DataStructure) {
val task = ui.newTask()
try {
task.header("Editing and Polishing Content")
// Step 1: Analyze the content to extract concepts, facts, and relationships
task.add("Analyzing the content for the topic: ${content.topic}")
val contentAnalysisResult = contentAnalyzerActor.answer(listOf(content.topic), api = api).obj
task.add("Content analysis complete. Extracted concepts and facts: ${contentAnalysisResult.concepts}")
// Step 2: Generate question stems based on the analyzed content
task.add("Generating question stems...")
val questionStemsResult = questionGeneratorActor.answer(listOf(content.topic), api = api).obj
task.add("Question stems generated: ${questionStemsResult.stems}")
// Step 3: Generate distractors for each question stem
task.add("Generating distractors for each question stem...")
val distractorsResult = questionStemsResult.stems.map { stem ->
stem to distractorGeneratorActor.answer(listOf(stem), api = api).obj.distractorMap[stem]
}.toMap()
task.add("AgentFlowDesignActors.Distractors generated for all question stems.")
// Step 4: Validate the generated questions and distractors
task.add("Validating questions...")
val validatedQuestions = distractorsResult.map { (stem, distractors) ->
validatorActor.answer(listOf(stem, distractors.toString()), api = api).obj
}
task.add("Questions validated.")
// Step 5: Adjust the difficulty level of the questions (if needed)
// This step would involve interaction with the user to set the desired difficulty level
// and then using the difficultyAdjusterActor to adjust the questions accordingly.
// Step 6: Compile the questions into a database and present to the user
// This step would involve storing the validated and adjusted questions into a database
// and providing the user with a link to download or interact with the database.
task.complete("Editing and polishing of content is complete. You can now review and download the questions.")
} catch (e: Throwable) {
task.error(e)
throw e
}
}
companion object {
private val log = org.slf4j.LoggerFactory.getLogger(AgentFlowDesignAgent::class.java)
}
}
class AgentFlowDesignActors(
val model: ChatModels = ChatModels.GPT4Turbo,
val temperature: Double = 0.3,
) {
interface ContentAnalysisParser : Function<String, ContentAnalysisResult> {
@Description("Parse the text into a structured content analysis result.")
override fun apply(text: String): ContentAnalysisResult
}
data class ContentAnalysisResult(
@Description("The key concepts identified in the content.")
val concepts: List<String>? = null,
@Description("The key facts identified in the content.")
val facts: List<String>? = null,
@Description("The relationships identified between concepts in the content.")
val relationships: List<String>? = null
) : ValidatedObject {
override fun validate() = when {
concepts.isNullOrEmpty() -> "At least one concept is required"
facts.isNullOrEmpty() -> "At least one fact is required"
relationships.isNullOrEmpty() -> "At least one relationship is required"
else -> null
}
}
val contentAnalyzerActor = ParsedActor<ContentAnalysisResult>(
parserClass = ContentAnalysisParser::class.java,
model = ChatModels.GPT35Turbo,
prompt = """
You are an assistant that analyzes educational content. Your task is to identify and list the key concepts, facts, and relationships present in the content provided.
Content:
""".trimIndent()
)
data class QuestionStems(
@Description("A list of generated question stems.")
val stems: List<String>
) : ValidatedObject {
override fun validate() = when {
stems.isEmpty() -> "At least one question stem is required"
stems.any { it.isBlank() } -> "Question stems cannot be blank"
else -> null
}
}
interface QuestionStemsParser : Function<String, QuestionStems> {
@Description("Parse the text response into a list of question stems.")
override fun apply(text: String): QuestionStems
}
val questionGeneratorActor = ParsedActor<QuestionStems>(
parserClass = QuestionStemsParser::class.java,
model = ChatModels.GPT35Turbo,
prompt = """
You are an assistant that generates multiple-choice question stems. Given a topic or material, produce a list of question stems that are clear, concise, and directly related to the topic. Ensure that the questions are open-ended and suitable for multiple-choice answers.
Topic: "The life cycle of a butterfly"
Generate question stems:
""".trimIndent()
)
data class Distractors(
@Description("A map of question stems to their respective list of distractors.")
val distractorMap: Map<String, List<String>>
) : ValidatedObject {
override fun validate() = when {
distractorMap.isEmpty() -> "distractorMap cannot be empty"
distractorMap.any { it.value.isEmpty() } -> "Each question stem must have at least one distractor"
else -> null
}
}
interface DistractorParser : Function<String, Distractors> {
@Description("Parse the text response into a map of question stems to their respective list of distractors.")
override fun apply(text: String): Distractors
}
val distractorGeneratorActor = ParsedActor<Distractors>(
parserClass = DistractorParser::class.java,
model = ChatModels.GPT35Turbo,
prompt = """
You are an assistant that generates plausible distractors for multiple-choice questions.
Given a question stem, produce a list of incorrect answers that are thematically consistent but clearly distinct from the correct answer.
Here is the question stem: [Question Stem]
Please provide a list of distractors.
""".trimIndent()
)
// Define the data class that will hold the validation result
data class ValidationResult(
@Description("The status indicating if the question is valid.")
val isValid: Boolean,
@Description("The feedback message explaining why the question is valid or not.")
val feedback: String? = null
) : ValidatedObject {
override fun validate() = when {
feedback.isNullOrBlank() -> "Feedback message is required"
else -> null
}
}
// Define the parser interface
interface ValidatorParser : Function<String, ValidationResult> {
@Description("Parse the text response into a ValidationResult.")
override fun apply(text: String): ValidationResult
}
// Instantiate the ParsedActor with the ValidatorParser
val validatorActor = ParsedActor<ValidationResult>(
parserClass = ValidatorParser::class.java,
model = ChatModels.GPT35Turbo,
prompt = """
You are an assistant that validates the correctness of multiple-choice questions.
Given a question with its correct answer and distractors, determine if the question is valid.
Provide feedback on the validity of the question.
""".trimIndent()
)
// Define the data class for the adjusted questions
data class AdjustedQuestion(
val stem: String,
val options: List<String>,
val correct: Int,
val difficulty: String
) : ValidatedObject {
override fun validate() = when {
stem.isBlank() -> "Question stem is required"
options.isEmpty() -> "Options are required"
correct !in options.indices -> "Correct answer index is out of bounds"
difficulty.isBlank() -> "Difficulty level is required"
else -> null
}
}
data class AdjustedQuestions(
val questions: List<AdjustedQuestion>
) : ValidatedObject {
override fun validate() = when {
questions.any { it.validate() != null } -> "All questions must be valid"
else -> null
}
}
// Define the parser interface
interface AdjustedQuestionsParser : Function<String, AdjustedQuestions> {
override fun apply(text: String): AdjustedQuestions
}
// Instantiate the difficultyAdjusterActor
val difficultyAdjusterActor = ParsedActor<AdjustedQuestions>(
parserClass = AdjustedQuestionsParser::class.java,
model = ChatModels.GPT35Turbo,
prompt = """
You are an assistant that adjusts the difficulty level of multiple-choice questions.
Given a list of questions and a specified difficulty level, modify the questions to match the desired difficulty.
Output the adjusted questions in a structured format.
""".trimIndent()
)
enum class ActorType {
CONTENT_ANALYZER_ACTOR,
QUESTION_GENERATOR_ACTOR,
DISTRACTOR_GENERATOR_ACTOR,
VALIDATOR_ACTOR,
DIFFICULTY_ADJUSTER_ACTOR,
}
val actorMap: Map<ActorType, BaseActor<out Any,out Any>> = mapOf(
ActorType.CONTENT_ANALYZER_ACTOR to contentAnalyzerActor,
ActorType.QUESTION_GENERATOR_ACTOR to questionGeneratorActor,
ActorType.DISTRACTOR_GENERATOR_ACTOR to distractorGeneratorActor,
ActorType.VALIDATOR_ACTOR to validatorActor,
ActorType.DIFFICULTY_ADJUSTER_ACTOR to difficultyAdjusterActor,
)
companion object {
val log = org.slf4j.LoggerFactory.getLogger(AgentFlowDesignActors::class.java)
}
}