This is a series of tasks I gave GitHub Copilot to work on in July 2024, to see if it would save time for the experienced software developers who work at Sembit. The tasks are performed on a simple internal web site we created to track incoming leads. These were tasks I had already completed myself. I reset my working copy of the source code to just before the relevant check-in I had made, and then asked then asked Copilot to do the task I had performed. I had done the same exercises a year earlier with Copilot X, and I wanted to see if the performance was any better or worse. Nobody knows if AI is going to hit a brick wall again and just inch forward, or if it's going to improve by leaps and bounds. In this particular case, I found the performance to be about the same as it was a year ago. As always, the specific details are far more instructive than generalizations, so here they are!
You can add the new link for leads that require followup after the "Open" link and before the "Closed" link by inserting a new anchor tag <a>
in the appropriate position within the div with id list-links. Here's how you can do it:
Please replace "LeadList?listCategory=Followup" with the actual URL that handles the followup leads.
Put the link after the Open link but before the Closed link.
You can add the new link for leads that require followup after the "Open" link and before the "Closed" link by inserting a new anchor tag <a>
in the appropriate position within the div with id list-links. Here's how you can do it:
Please replace "LeadList?listCategory=Followup" with the actual URL that handles the followup leads.
Add a new URL to a menu
Excellent
A little worse than last year. Last year it chose display text as "Follow Up" while keeping the querystring as "Followup", which was a great decision. This year, it used "Followup" for the English display as well as the querystring. However, in both cases it guessed a very appropriate querystring value, could understand my request for a change in relative position, and emitted perfect HTML syntax.
Change lead activity to allow multiline input and display multiline output in the activity log. (Try #1)
I highlighted the entire HTML file, 236 lines. Relevant lines shown here:
Change lead activity data entry so that it supports multiline input and displays multiline output in the activity log.
To support multiline input and display multiline output in the activity log, you need to change the input field for lead activity data entry from a single-line text input to a textarea.
Here's how you can do it:
1. Find the input field for lead activity data entry in your `Lead.cshtml` file. It might look something like this:
3. To display multiline output in the activity log, ensure that the output is wrapped in a block-level element like a `<p>
` or `<div>
`, and that line breaks are preserved. This can be done using CSS.
.activity-log {
white-space: pre-line;
}
This CSS rule will preserve line breaks in the text, so if the user enters text with line breaks into the textarea, those line breaks will be displayed in the activity log.
Remember to replace `leadActivity` and `activity-log` with the actual IDs or classes used in your HTML.
Change lead activity to allow multiline input and display multiline output in the activity log. (Try #1)
OK, if there was no existing code.
It had a general idea but gave a non-specific answer. Apparently it was too difficult to identify what I was talking about in the entire HTML file. However, this is much better than my test a year ago where I did not give it context, where it completely misunderstood the request and said something about modifying the VS code integrated terminal settings.
Change lead activity to allow multiline input and display multiline output in the activity log. (Try #2)
I highlighted just the section with the edit and the readonly log, 64 lines of HTML. Relevant lines shown here:
<h2>Activity Log</h2>
<div id="activity">
@foreach(var a in Model.Activity) {
<div><span class="activity-date">@(a.ActivityDate.ToString("ddd MMM d"))</span> @a.ActivityNote</div>
}
</div>
Change lead activity data entry so that it supports multiline input and displays multiline output in the activity log.
To support multiline input, you can change the input type from "text" to "textarea" for the activity note. For displaying multiline output in the activity log, you can replace the div with a preformatted text tag (`<pre>`).
Here's how you can modify your code:
<div id="new-activity-edit">
<div>New Activity: @Html.Date("NewActivity.ActivityDate", Model.NewActivity.ActivityDate)
<textarea id="new-activity-note" name="NewActivity.ActivityNote" rows="4" cols="50"></textarea></div> @* Use textarea to support multiline input. *@
<!-- Rest of your code -->
</div>
<h2>Activity Log</h2>
<div id="activity">
@foreach(var a in Model.Activity) {
<pre class="activity-date">@(a.ActivityDate.ToString("ddd MMM d")) @a.ActivityNote</pre> @* Use preformatted text to display multiline output. *@
}
</div>
This will allow users to enter multiline notes for the activity and will display these notes in the activity log preserving the line breaks.
Change lead activity to allow multiline input and display multiline output in the activity log. (Try #2)
Very good.
This time it gave specific changes that were very reasonable solutions to the problem it was given. Narrowing the context made a huge improvement in its performance. Similar to its performance last year - good once I highlighted the exact subset of HTML that needed to be changed.
Change format of new activity label / date input / note input, currently all in a horizontal row, so note is directly beneath date input.
Move the activity note textarea underneath the new activity date field, so the left edge of the textarea is aligned with the left edge of the activity date field.
You can achieve this by wrapping each field in a separate `div` element. This will ensure that each field starts on a new line and their left edges are aligned. Here's how you can modify your code:
This will place the textarea underneath the date field, aligning their left edges.
No, that would left-align the textarea with the label. I want the textarea even with the date field. Let's ask for a correction.
No, that aligns the textarea to the left of the "New Activity" label. I want the textarea and the input created by the @Html.Date function to be aligned with each other.
I see, you want the textarea to align with the date input field, not the label. You can achieve this by wrapping the label and the input field in separate `div` elements. Here's how you can modify your code:
This will place the "New Activity" label to the left of the ActivityDate. The textarea for the Activity Note will be placed below the ActivityDate.
This is getting closer to a useful solution, but it should have just the label in the first column, and the two fields in the second column.
Please rearrange the columns such that the New Activity label is in the first column, and the date and note are in the second column, so that the note will be directly below the date.
To achieve this, you can modify your HTML structure and use CSS Flexbox. Here's how you can modify your code:
This will place the "New Activity" label in the first column, and the date and note in the second column, with the note directly below the date.
Yes! This achieves what I wanted.
Change format of new activity label / date input / note input, currently all in a horizontal row, so note is directly beneath date input.
Good, eventually, after 3 rounds of restatement and clarification.
This felt painful. It was also worse than last year, where on the first try it went to a flexbox layout with two columns. Now, last year wasn't perfect - it also inserted extra labels in the second column, which I had to tell it to remove.
Create a new table in a database, given a description of what I want to track.
Having learned from last year, I highlighted an example table as input.
create table LeadActivity (
LeadActivityID int not null primary key identity(1,1),
LeadID int not null,
CreateTime datetime2 not null, --Don't bother displaying this
ActivityDate date not null,
ActivityNote varchar(max) not null
)
go
Create a new table to track when someone visits a Google Ad Landing page. I want to track when the page loads, the name of the landing page, the gclid (which should have an index), the user's IP address, the useragent string, and the referrer. Generate a create table statement in the same style as the input I have given you.
Here is the SQL statement to create a new table to track when someone visits a Google Ad Landing page:
create table GoogleAdLandingPageVisit (
VisitID int not null primary key identity(1,1),
PageLoadTime datetime2 not null,
LandingPageName varchar(255) not null,
GCLID varchar(255) not null,
UserIPAddress varchar(45) not null, -- IPv4 and IPv6 support
UserAgentString varchar(max) not null,
Referrer varchar(max) not null
)
go
create index idx_GoogleAdLandingPageVisit_GCLID
on GoogleAdLandingPageVisit (GCLID)
go
This SQL statement creates a new table named `GoogleAdLandingPageVisit` with the fields you specified. It also creates an index on the `GCLID` field for faster queries.
Create a new table in a database, given a description of what I want to track.
Very good.
This is a correct create table statement, the column names are pretty close to what I wanted, and the max length of the varchar fields is reasonable. It is not quite as good as my final attempt last year in terms of picking up on the naming and varchar max lengths, perhaps because this time I gave it a different example table with fewer columns.
Create a function to log a record to the new table. (Try #1)
I highlighted the create table script, and attempted unsuccessfully to reference the existing Query.cs file using the #file directive.
create table GoogleAdLanding (
GoogleAdLandingID int not null primary key identity(1,1),
LoadTime datetime2 not null,
LandingPage varchar(100) not null, --Sembit landing page name, not including any parameters
gclid varchar(100) not null,
IPAddress varchar(100) not null,
UserAgent varchar(200) not null,
Referrer varchar(200) not null
)
go
create index xgclid on GoogleAdLanding(gclid)
go
In #file:Query.cs create a new C# function called RecordGoogleAdLanding which takes an http request parameter, a landingpage parameter, and a gclid parameter. It should get the IP address and useragent and referrer from the request, and insert all of these into the GoogleAdLandingPageVisit table represented by the given create table statement.
Here is the pseudocode for the function:
1. Define a function called `RecordGoogleAdLanding` that takes three parameters: an `HttpRequest` object, a `landingPage` string, and a `gclid` string.
2. From the `HttpRequest` object, extract the IP address, user agent, and referrer.
3. Create a SQL command string that inserts the `landingPage`, `gclid`, IP address, user agent, and referrer into the `GoogleAdLanding` table. The `LoadTime` should be the current time.
4. Execute the SQL command.
Here is the C# code:
using System;
using System.Data.SqlClient;
using Microsoft.AspNetCore.Http;
public void RecordGoogleAdLanding(HttpRequest request, string landingPage, string gclid)
{
string ipAddress = request.HttpContext.Connection.RemoteIpAddress.ToString();
string userAgent = request.Headers["User-Agent"].ToString();
string referrer = request.Headers["Referer"].ToString();
string connectionString = "YourConnectionString"; // replace with your connection string
using (SqlConnection connection = new SqlConnection(connectionString))
{
connection.Open();
string sql = "INSERT INTO GoogleAdLanding (LoadTime, LandingPage, gclid, IPAddress, UserAgent, Referrer) " +
"VALUES (@LoadTime, @LandingPage, @gclid, @IPAddress, @UserAgent, @Referrer)";
using (SqlCommand command = new SqlCommand(sql, connection))
{
command.Parameters.AddWithValue("@LoadTime", DateTime.Now);
command.Parameters.AddWithValue("@LandingPage", landingPage);
command.Parameters.AddWithValue("@gclid", gclid);
command.Parameters.AddWithValue("@IPAddress", ipAddress);
command.Parameters.AddWithValue("@UserAgent", userAgent);
command.Parameters.AddWithValue("@Referrer", referrer);
command.ExecuteNonQuery();
}
}
}
Please replace `"YourConnectionString"` with your actual database connection string. This function assumes that you are using SQL Server and the `System.Data.SqlClient` namespace for database operations. If you are using a different database system or library, the code may need to be adjusted accordingly.
Create a function to log a record to the new table. (Try #1)
OK, if there was no framework or existing code.
I tried a few variations, but was not able to get the #file directive to work. Without context, the AI gave an accurate but inappropriate answer.
Create a function to log a record to the new table. (Try #2)
This time I put all the context in one huge prompt that included my instructions, the create table statement, and an example function.
Create a new C# function called RecordGoogleAdLanding which takes an http request parameter, a landingpage parameter, and a gclid parameter. It should get the IP address and useragent and referrer from the request, and insert all of these into the GoogleAdLandingPageVisit table represented by this create table statement:
create table GoogleAdLanding (
GoogleAdLandingID int not null primary key identity(1,1),
LoadTime datetime2 not null,
LandingPage varchar(100) not null, --Sembit landing page name, not including any parameters
gclid varchar(100) not null,
IPAddress varchar(100) not null,
UserAgent varchar(200) not null,
Referrer varchar(200) not null
)
go
Make the function in the style of the following function, using the vs and Max functions as shown, but without the try/catch clause:
public void RecordGoogleAdClick(string landingPage,
string gclid, string keyword, string placement, string target, string targetid,
string creative, string campaignid, string adgroupid, string network, string device,
string loc_interest_ms, string loc_physical_ms, string matchtype) {
string sql = @"insert GoogleAdClick (ClickTime, LandingPage,
gclid, keyword, placement, target, targetid,
creative, campaignid, adgroupid, network, device,
loc_interest_ms, loc_physical_ms, matchtype)
select sysdatetime(), " + vs(Max(landingPage, 100),
Max(gclid, 100), Max(keyword, 500), Max(placement, 500), Max(target, 500), Max(targetid, 500),
Max(creative,100), Max(campaignid, 100), Max(adgroupid, 100), Max(network, 5), Max(device, 5),
Max(loc_interest_ms, 10), Max(loc_physical_ms, 10), Max(matchtype, 5));
try {
db.Execute(sql);
} catch(Exception x) {
//Swallow duplicate PK error (in case double-click an ad), but re-throw anything else
if(!x.Message.Contains("Cannot insert duplicate key"))
//Not a duplicate PK error, so abort
throw;
}
}
Here is the new function `RecordGoogleAdLanding` in C#:
This function takes an `HttpRequest` object, a `landingPage` string, and a `gclid` string as parameters. It extracts the IP address, user agent, and referrer from the request. Then it constructs an SQL insert statement and executes it. The `vs` and `Max` functions are used to ensure the lengths of the string parameters do not exceed the maximum lengths defined in the database schema.
Create a function to log a record to the new table. (Try #2)
Excellent The SQL string is constructed exactly as I wanted it, and the mapping from the request object is close.
In my actual code, I used some other properties of the HttpRequestBase class, but this code would work. I did a double-take when I saw "Referer" as the name of an HTTP header, but I looked it up and found that it's correct - the HTTP spec and implementation has that spelling error in it.
When I think about the fact that a machine was able to respond to my requests and actually produce code that is good or even excellent, I am blown away. This AI technology is completely amazing. Even when it's wrong, it gets the syntax right.
On the other hand, when I try to imagine delegating this task like I would to another developer, it seems frustrating. (Of course, it's unlikely I would hand off such tiny tasks in the real world - it would be stated more broadly. But the current tool is certainly not capable of broader instruction given its limited context capabilities.)
If I imagine telling a junior developer to use GitHub Copilot, and I expect that developer to be able to know when they've gotten a good result, that seems problematic. I fear they would skip understanding what's going on and just try to rephrase their prompt, running the program until it seems to work. If so, they would never gain mastery of the underlying concepts or technology. Good developers understand the language and framework so they can determine whether a given implementation is appropriate.
As a senior developer, it feels frustrating to try and code via this extra layer of indirection. Instead of directly translating my ideas to code, I have to explain what I want, then if I don't get lucky the first time I have to wrack my brain to figure out how to get the AI to adjust its output. It's actually more annoying and probably slower than dictating the code directly.
Overall, this isn't a tool we're going to find much use for at Sembit in its current state, as our developers are very experienced. However, if you have a lot of coding tasks which call for following a pattern in a fairly mechanical way that isn't quite consistent enough to be created via code generation, this might actually save you time. Hopefully this article has given you a taste for what you might encounter, so you can make an informed decision about the state of AI for programming and whether you might care to use it.
