Posted Wednesday, April 17, 2013 in Old JamesCMS Posts
More websockets for jamescms. This time I built a trivia game similar to some irc bots out there. I can't seem to find the link now but I downloaded a text file database of questions for this. There turned out to be a total of 416,425 questions all of which I uploaded into my sql server. Each question in the text files were deliminated by an asterix. I used .net's stream reader to read through the files and upload each question.
{{C#}}
QuizGameContext qg = new QuizGameContext();
qg.Configuration.AutoDetectChangesEnabled = false;
string filepath = "D:\\Downloads\\trivia";
foreach (var file in Directory.GetFiles(filepath))
{
using (StreamReader fr = new StreamReader(file))
{
string line = "";
int lineCount = 0;
while ((line = fr.ReadLine()) != null)
{
lineCount++;
var parts = line.Split('*');
if (parts.Length == 2 &&
!string.IsNullOrEmpty(parts[0]) &&
!string.IsNullOrEmpty(parts[1]))
{
TriviaQuestion temp = new TriviaQuestion()
{
Question = parts[0],
Answer = parts[1]
};
qg.TriviaQuestions.Add(temp);
try
{
if (lineCount % 100 == 0)
{
qg.SaveChanges();
qg.Dispose();
qg = new QuizGameContext();
qg.Configuration.AutoDetectChangesEnabled = false;
}
}
catch { }
}
}
}
}
My first attempt didn't include the dispose of the quiz game context. My first attempt also failed at about 80k inserts. This question helped me out. After adding the dispose the entire 400k+ inserts took about a minute compared to the 30 minutes the first attempt took that ended in a memory access issue.