Xcode UITesting test for enabled segmented control fails in iOS12
up vote
1
down vote
favorite
Before I submitted this to Apple as a bug report, I wanted to double check I am not doing something very silly.
I have attached a sample project that demonstrates the problem. I have two segmented controls, one of which controls the enabled state of the other.
https://www.dropbox.com/s/dq2x9srbme4genb/EnabledControlsProblem.zip?dl=0
If you click on the disabled button, it disabled the second segmented control.
I have a basic UI test that replicates this behaviour which is constructed like this:
XCTAssertTrue(app.segmentedControls.buttons["Enabled"].exists)
XCTAssertTrue(app.segmentedControls.buttons["Enabled"].isSelected)
XCTAssertTrue(app.segmentedControls.buttons["First"].exists)
XCTAssertTrue(app.segmentedControls.buttons["First"].isSelected)
XCTAssertFalse(app.segmentedControls.buttons["Second"].isSelected)
XCTAssertTrue(app.segmentedControls.containing(.button, identifier: "First").firstMatch.isEnabled)
app.segmentedControls.buttons["Disabled"].tap()
XCTAssertFalse(app.segmentedControls.containing(.button, identifier: "First").firstMatch.isEnabled)`
If you run this test on an iOS 11.4 device it works as I think it should and succeeds. However, if I switch to a iOS 12.1 simulator device, the test fails on the last line (which is checking if the second control is enabled). Visually state of the control changes as it should. I have also tried this same test on an actual iPhone device running 12.0
Any attempts to sleep before the last check, don't make a difference.
Is there something fundamentally wrong with what I have done or is this a bug or a change in iOS12 that I missed?
ios xcode xcode-ui-testing
add a comment |
up vote
1
down vote
favorite
Before I submitted this to Apple as a bug report, I wanted to double check I am not doing something very silly.
I have attached a sample project that demonstrates the problem. I have two segmented controls, one of which controls the enabled state of the other.
https://www.dropbox.com/s/dq2x9srbme4genb/EnabledControlsProblem.zip?dl=0
If you click on the disabled button, it disabled the second segmented control.
I have a basic UI test that replicates this behaviour which is constructed like this:
XCTAssertTrue(app.segmentedControls.buttons["Enabled"].exists)
XCTAssertTrue(app.segmentedControls.buttons["Enabled"].isSelected)
XCTAssertTrue(app.segmentedControls.buttons["First"].exists)
XCTAssertTrue(app.segmentedControls.buttons["First"].isSelected)
XCTAssertFalse(app.segmentedControls.buttons["Second"].isSelected)
XCTAssertTrue(app.segmentedControls.containing(.button, identifier: "First").firstMatch.isEnabled)
app.segmentedControls.buttons["Disabled"].tap()
XCTAssertFalse(app.segmentedControls.containing(.button, identifier: "First").firstMatch.isEnabled)`
If you run this test on an iOS 11.4 device it works as I think it should and succeeds. However, if I switch to a iOS 12.1 simulator device, the test fails on the last line (which is checking if the second control is enabled). Visually state of the control changes as it should. I have also tried this same test on an actual iPhone device running 12.0
Any attempts to sleep before the last check, don't make a difference.
Is there something fundamentally wrong with what I have done or is this a bug or a change in iOS12 that I missed?
ios xcode xcode-ui-testing
add a comment |
up vote
1
down vote
favorite
up vote
1
down vote
favorite
Before I submitted this to Apple as a bug report, I wanted to double check I am not doing something very silly.
I have attached a sample project that demonstrates the problem. I have two segmented controls, one of which controls the enabled state of the other.
https://www.dropbox.com/s/dq2x9srbme4genb/EnabledControlsProblem.zip?dl=0
If you click on the disabled button, it disabled the second segmented control.
I have a basic UI test that replicates this behaviour which is constructed like this:
XCTAssertTrue(app.segmentedControls.buttons["Enabled"].exists)
XCTAssertTrue(app.segmentedControls.buttons["Enabled"].isSelected)
XCTAssertTrue(app.segmentedControls.buttons["First"].exists)
XCTAssertTrue(app.segmentedControls.buttons["First"].isSelected)
XCTAssertFalse(app.segmentedControls.buttons["Second"].isSelected)
XCTAssertTrue(app.segmentedControls.containing(.button, identifier: "First").firstMatch.isEnabled)
app.segmentedControls.buttons["Disabled"].tap()
XCTAssertFalse(app.segmentedControls.containing(.button, identifier: "First").firstMatch.isEnabled)`
If you run this test on an iOS 11.4 device it works as I think it should and succeeds. However, if I switch to a iOS 12.1 simulator device, the test fails on the last line (which is checking if the second control is enabled). Visually state of the control changes as it should. I have also tried this same test on an actual iPhone device running 12.0
Any attempts to sleep before the last check, don't make a difference.
Is there something fundamentally wrong with what I have done or is this a bug or a change in iOS12 that I missed?
ios xcode xcode-ui-testing
Before I submitted this to Apple as a bug report, I wanted to double check I am not doing something very silly.
I have attached a sample project that demonstrates the problem. I have two segmented controls, one of which controls the enabled state of the other.
https://www.dropbox.com/s/dq2x9srbme4genb/EnabledControlsProblem.zip?dl=0
If you click on the disabled button, it disabled the second segmented control.
I have a basic UI test that replicates this behaviour which is constructed like this:
XCTAssertTrue(app.segmentedControls.buttons["Enabled"].exists)
XCTAssertTrue(app.segmentedControls.buttons["Enabled"].isSelected)
XCTAssertTrue(app.segmentedControls.buttons["First"].exists)
XCTAssertTrue(app.segmentedControls.buttons["First"].isSelected)
XCTAssertFalse(app.segmentedControls.buttons["Second"].isSelected)
XCTAssertTrue(app.segmentedControls.containing(.button, identifier: "First").firstMatch.isEnabled)
app.segmentedControls.buttons["Disabled"].tap()
XCTAssertFalse(app.segmentedControls.containing(.button, identifier: "First").firstMatch.isEnabled)`
If you run this test on an iOS 11.4 device it works as I think it should and succeeds. However, if I switch to a iOS 12.1 simulator device, the test fails on the last line (which is checking if the second control is enabled). Visually state of the control changes as it should. I have also tried this same test on an actual iPhone device running 12.0
Any attempts to sleep before the last check, don't make a difference.
Is there something fundamentally wrong with what I have done or is this a bug or a change in iOS12 that I missed?
ios xcode xcode-ui-testing
ios xcode xcode-ui-testing
asked Nov 9 at 18:53
spottedrabbit
382211
382211
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
up vote
0
down vote
I used to have the same type of problems with UI testing and I discovered a better method in pretty much all cases, and this lies in the key point that all tests rely on the Accessibility Framework. I was getting these weird seemingly buggy results all the time.
1. Better Debugging
Debugging goes much easier when you use the following command, which will show the hierarchy of elements:
print("Current App Tree: (app.debugDescription)")
The hierarchy will help you ensure that the element does exists and how the app/device/simulator is seeing it.
2. Use Accessibility Identifiers
Instead of finding elements as you are, understand that all the tests are built on the accessibility framework. This seems insignificant at first, but I found it made life much easier. The framework is much happier when all elements have an accessibility identifier. Basically each view has a property: "accessibilityIdentifier". This will show up in the app debug listed above. If you do not set this, the framework will have a hard time finding your elements. If you're using storyboards, this is easy to set under Accessibility. If you're using code, set the property as follows:
myView.accessibilityIdentifier = "Enable State Selection"
Then in your UI Tests you can find the element like this:
app.buttons.matching(identifier: "Enable State Selection").element
In your specific case, it looks like it is part of a UI group of some kind involved in segmented controls. I would make a two step finding like this:
myUIGroup.accessibilityIdentifier = "Controls"
Then you can do a two step finding like this:
let controlsView = app.segmentedControls.matching(identifier: "Controls").element
let enabledButtons = controlsView.buttons.matching(identifier: "Enable State Selection").element
XCTAssert(enabledButtons.exists)
Overall I find that you at the beginning of your UI Testing class define these as class globals:
let app = XCUIApplication()
lazy var controlsView = app.segmentedControls.matching(identifier: "Controls").element
lazy var enabledButtons = controlsView.buttons.matching(identifier: "Enable State Selection").element
Then in your test function the code is very clean, and simply:
XCTAssert(enabledButtons.exists)
Thanks for the response, but I am not sure its that simple in this case. While I didn't set the accessibiliyId for the segmented controls, setting it explicitly does not change the result I am seeing.
– spottedrabbit
Nov 10 at 11:57
Are you saying it doesn't show up in the debugDescription? That's the first step. The UI elements can be moved around I'm not sure how querying works exactly, but it's not always consistent. If you add the identifier does it show up in your debugDescription with the identifier? That's the first step.
– David J
Nov 10 at 12:00
The segmented control shows up when you call debugPrint(app) and either shows or doesn't show the identifier depending on if it is set. I don't think this a problem of the UIElement not being in the view hierarchy correctly.
– spottedrabbit
Nov 10 at 12:04
Yes, unfortunately, the view hierarchy may change based upon certain conditions, which is I had to give up screen recording and really learn the framework. That's why the identifiers is important and why you may seen different results in different OSes and devices. If you set the identifier then you can guarantee you can find it. It also helps to have multiple levels rather than starting each search from the app. You can also print out the debug of each element you find as opposed to the entire app. print("(controlsView.debugDescription)"). I hope it helps.
– David J
Nov 10 at 12:16
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
0
down vote
I used to have the same type of problems with UI testing and I discovered a better method in pretty much all cases, and this lies in the key point that all tests rely on the Accessibility Framework. I was getting these weird seemingly buggy results all the time.
1. Better Debugging
Debugging goes much easier when you use the following command, which will show the hierarchy of elements:
print("Current App Tree: (app.debugDescription)")
The hierarchy will help you ensure that the element does exists and how the app/device/simulator is seeing it.
2. Use Accessibility Identifiers
Instead of finding elements as you are, understand that all the tests are built on the accessibility framework. This seems insignificant at first, but I found it made life much easier. The framework is much happier when all elements have an accessibility identifier. Basically each view has a property: "accessibilityIdentifier". This will show up in the app debug listed above. If you do not set this, the framework will have a hard time finding your elements. If you're using storyboards, this is easy to set under Accessibility. If you're using code, set the property as follows:
myView.accessibilityIdentifier = "Enable State Selection"
Then in your UI Tests you can find the element like this:
app.buttons.matching(identifier: "Enable State Selection").element
In your specific case, it looks like it is part of a UI group of some kind involved in segmented controls. I would make a two step finding like this:
myUIGroup.accessibilityIdentifier = "Controls"
Then you can do a two step finding like this:
let controlsView = app.segmentedControls.matching(identifier: "Controls").element
let enabledButtons = controlsView.buttons.matching(identifier: "Enable State Selection").element
XCTAssert(enabledButtons.exists)
Overall I find that you at the beginning of your UI Testing class define these as class globals:
let app = XCUIApplication()
lazy var controlsView = app.segmentedControls.matching(identifier: "Controls").element
lazy var enabledButtons = controlsView.buttons.matching(identifier: "Enable State Selection").element
Then in your test function the code is very clean, and simply:
XCTAssert(enabledButtons.exists)
Thanks for the response, but I am not sure its that simple in this case. While I didn't set the accessibiliyId for the segmented controls, setting it explicitly does not change the result I am seeing.
– spottedrabbit
Nov 10 at 11:57
Are you saying it doesn't show up in the debugDescription? That's the first step. The UI elements can be moved around I'm not sure how querying works exactly, but it's not always consistent. If you add the identifier does it show up in your debugDescription with the identifier? That's the first step.
– David J
Nov 10 at 12:00
The segmented control shows up when you call debugPrint(app) and either shows or doesn't show the identifier depending on if it is set. I don't think this a problem of the UIElement not being in the view hierarchy correctly.
– spottedrabbit
Nov 10 at 12:04
Yes, unfortunately, the view hierarchy may change based upon certain conditions, which is I had to give up screen recording and really learn the framework. That's why the identifiers is important and why you may seen different results in different OSes and devices. If you set the identifier then you can guarantee you can find it. It also helps to have multiple levels rather than starting each search from the app. You can also print out the debug of each element you find as opposed to the entire app. print("(controlsView.debugDescription)"). I hope it helps.
– David J
Nov 10 at 12:16
add a comment |
up vote
0
down vote
I used to have the same type of problems with UI testing and I discovered a better method in pretty much all cases, and this lies in the key point that all tests rely on the Accessibility Framework. I was getting these weird seemingly buggy results all the time.
1. Better Debugging
Debugging goes much easier when you use the following command, which will show the hierarchy of elements:
print("Current App Tree: (app.debugDescription)")
The hierarchy will help you ensure that the element does exists and how the app/device/simulator is seeing it.
2. Use Accessibility Identifiers
Instead of finding elements as you are, understand that all the tests are built on the accessibility framework. This seems insignificant at first, but I found it made life much easier. The framework is much happier when all elements have an accessibility identifier. Basically each view has a property: "accessibilityIdentifier". This will show up in the app debug listed above. If you do not set this, the framework will have a hard time finding your elements. If you're using storyboards, this is easy to set under Accessibility. If you're using code, set the property as follows:
myView.accessibilityIdentifier = "Enable State Selection"
Then in your UI Tests you can find the element like this:
app.buttons.matching(identifier: "Enable State Selection").element
In your specific case, it looks like it is part of a UI group of some kind involved in segmented controls. I would make a two step finding like this:
myUIGroup.accessibilityIdentifier = "Controls"
Then you can do a two step finding like this:
let controlsView = app.segmentedControls.matching(identifier: "Controls").element
let enabledButtons = controlsView.buttons.matching(identifier: "Enable State Selection").element
XCTAssert(enabledButtons.exists)
Overall I find that you at the beginning of your UI Testing class define these as class globals:
let app = XCUIApplication()
lazy var controlsView = app.segmentedControls.matching(identifier: "Controls").element
lazy var enabledButtons = controlsView.buttons.matching(identifier: "Enable State Selection").element
Then in your test function the code is very clean, and simply:
XCTAssert(enabledButtons.exists)
Thanks for the response, but I am not sure its that simple in this case. While I didn't set the accessibiliyId for the segmented controls, setting it explicitly does not change the result I am seeing.
– spottedrabbit
Nov 10 at 11:57
Are you saying it doesn't show up in the debugDescription? That's the first step. The UI elements can be moved around I'm not sure how querying works exactly, but it's not always consistent. If you add the identifier does it show up in your debugDescription with the identifier? That's the first step.
– David J
Nov 10 at 12:00
The segmented control shows up when you call debugPrint(app) and either shows or doesn't show the identifier depending on if it is set. I don't think this a problem of the UIElement not being in the view hierarchy correctly.
– spottedrabbit
Nov 10 at 12:04
Yes, unfortunately, the view hierarchy may change based upon certain conditions, which is I had to give up screen recording and really learn the framework. That's why the identifiers is important and why you may seen different results in different OSes and devices. If you set the identifier then you can guarantee you can find it. It also helps to have multiple levels rather than starting each search from the app. You can also print out the debug of each element you find as opposed to the entire app. print("(controlsView.debugDescription)"). I hope it helps.
– David J
Nov 10 at 12:16
add a comment |
up vote
0
down vote
up vote
0
down vote
I used to have the same type of problems with UI testing and I discovered a better method in pretty much all cases, and this lies in the key point that all tests rely on the Accessibility Framework. I was getting these weird seemingly buggy results all the time.
1. Better Debugging
Debugging goes much easier when you use the following command, which will show the hierarchy of elements:
print("Current App Tree: (app.debugDescription)")
The hierarchy will help you ensure that the element does exists and how the app/device/simulator is seeing it.
2. Use Accessibility Identifiers
Instead of finding elements as you are, understand that all the tests are built on the accessibility framework. This seems insignificant at first, but I found it made life much easier. The framework is much happier when all elements have an accessibility identifier. Basically each view has a property: "accessibilityIdentifier". This will show up in the app debug listed above. If you do not set this, the framework will have a hard time finding your elements. If you're using storyboards, this is easy to set under Accessibility. If you're using code, set the property as follows:
myView.accessibilityIdentifier = "Enable State Selection"
Then in your UI Tests you can find the element like this:
app.buttons.matching(identifier: "Enable State Selection").element
In your specific case, it looks like it is part of a UI group of some kind involved in segmented controls. I would make a two step finding like this:
myUIGroup.accessibilityIdentifier = "Controls"
Then you can do a two step finding like this:
let controlsView = app.segmentedControls.matching(identifier: "Controls").element
let enabledButtons = controlsView.buttons.matching(identifier: "Enable State Selection").element
XCTAssert(enabledButtons.exists)
Overall I find that you at the beginning of your UI Testing class define these as class globals:
let app = XCUIApplication()
lazy var controlsView = app.segmentedControls.matching(identifier: "Controls").element
lazy var enabledButtons = controlsView.buttons.matching(identifier: "Enable State Selection").element
Then in your test function the code is very clean, and simply:
XCTAssert(enabledButtons.exists)
I used to have the same type of problems with UI testing and I discovered a better method in pretty much all cases, and this lies in the key point that all tests rely on the Accessibility Framework. I was getting these weird seemingly buggy results all the time.
1. Better Debugging
Debugging goes much easier when you use the following command, which will show the hierarchy of elements:
print("Current App Tree: (app.debugDescription)")
The hierarchy will help you ensure that the element does exists and how the app/device/simulator is seeing it.
2. Use Accessibility Identifiers
Instead of finding elements as you are, understand that all the tests are built on the accessibility framework. This seems insignificant at first, but I found it made life much easier. The framework is much happier when all elements have an accessibility identifier. Basically each view has a property: "accessibilityIdentifier". This will show up in the app debug listed above. If you do not set this, the framework will have a hard time finding your elements. If you're using storyboards, this is easy to set under Accessibility. If you're using code, set the property as follows:
myView.accessibilityIdentifier = "Enable State Selection"
Then in your UI Tests you can find the element like this:
app.buttons.matching(identifier: "Enable State Selection").element
In your specific case, it looks like it is part of a UI group of some kind involved in segmented controls. I would make a two step finding like this:
myUIGroup.accessibilityIdentifier = "Controls"
Then you can do a two step finding like this:
let controlsView = app.segmentedControls.matching(identifier: "Controls").element
let enabledButtons = controlsView.buttons.matching(identifier: "Enable State Selection").element
XCTAssert(enabledButtons.exists)
Overall I find that you at the beginning of your UI Testing class define these as class globals:
let app = XCUIApplication()
lazy var controlsView = app.segmentedControls.matching(identifier: "Controls").element
lazy var enabledButtons = controlsView.buttons.matching(identifier: "Enable State Selection").element
Then in your test function the code is very clean, and simply:
XCTAssert(enabledButtons.exists)
answered Nov 10 at 10:26
David J
50638
50638
Thanks for the response, but I am not sure its that simple in this case. While I didn't set the accessibiliyId for the segmented controls, setting it explicitly does not change the result I am seeing.
– spottedrabbit
Nov 10 at 11:57
Are you saying it doesn't show up in the debugDescription? That's the first step. The UI elements can be moved around I'm not sure how querying works exactly, but it's not always consistent. If you add the identifier does it show up in your debugDescription with the identifier? That's the first step.
– David J
Nov 10 at 12:00
The segmented control shows up when you call debugPrint(app) and either shows or doesn't show the identifier depending on if it is set. I don't think this a problem of the UIElement not being in the view hierarchy correctly.
– spottedrabbit
Nov 10 at 12:04
Yes, unfortunately, the view hierarchy may change based upon certain conditions, which is I had to give up screen recording and really learn the framework. That's why the identifiers is important and why you may seen different results in different OSes and devices. If you set the identifier then you can guarantee you can find it. It also helps to have multiple levels rather than starting each search from the app. You can also print out the debug of each element you find as opposed to the entire app. print("(controlsView.debugDescription)"). I hope it helps.
– David J
Nov 10 at 12:16
add a comment |
Thanks for the response, but I am not sure its that simple in this case. While I didn't set the accessibiliyId for the segmented controls, setting it explicitly does not change the result I am seeing.
– spottedrabbit
Nov 10 at 11:57
Are you saying it doesn't show up in the debugDescription? That's the first step. The UI elements can be moved around I'm not sure how querying works exactly, but it's not always consistent. If you add the identifier does it show up in your debugDescription with the identifier? That's the first step.
– David J
Nov 10 at 12:00
The segmented control shows up when you call debugPrint(app) and either shows or doesn't show the identifier depending on if it is set. I don't think this a problem of the UIElement not being in the view hierarchy correctly.
– spottedrabbit
Nov 10 at 12:04
Yes, unfortunately, the view hierarchy may change based upon certain conditions, which is I had to give up screen recording and really learn the framework. That's why the identifiers is important and why you may seen different results in different OSes and devices. If you set the identifier then you can guarantee you can find it. It also helps to have multiple levels rather than starting each search from the app. You can also print out the debug of each element you find as opposed to the entire app. print("(controlsView.debugDescription)"). I hope it helps.
– David J
Nov 10 at 12:16
Thanks for the response, but I am not sure its that simple in this case. While I didn't set the accessibiliyId for the segmented controls, setting it explicitly does not change the result I am seeing.
– spottedrabbit
Nov 10 at 11:57
Thanks for the response, but I am not sure its that simple in this case. While I didn't set the accessibiliyId for the segmented controls, setting it explicitly does not change the result I am seeing.
– spottedrabbit
Nov 10 at 11:57
Are you saying it doesn't show up in the debugDescription? That's the first step. The UI elements can be moved around I'm not sure how querying works exactly, but it's not always consistent. If you add the identifier does it show up in your debugDescription with the identifier? That's the first step.
– David J
Nov 10 at 12:00
Are you saying it doesn't show up in the debugDescription? That's the first step. The UI elements can be moved around I'm not sure how querying works exactly, but it's not always consistent. If you add the identifier does it show up in your debugDescription with the identifier? That's the first step.
– David J
Nov 10 at 12:00
The segmented control shows up when you call debugPrint(app) and either shows or doesn't show the identifier depending on if it is set. I don't think this a problem of the UIElement not being in the view hierarchy correctly.
– spottedrabbit
Nov 10 at 12:04
The segmented control shows up when you call debugPrint(app) and either shows or doesn't show the identifier depending on if it is set. I don't think this a problem of the UIElement not being in the view hierarchy correctly.
– spottedrabbit
Nov 10 at 12:04
Yes, unfortunately, the view hierarchy may change based upon certain conditions, which is I had to give up screen recording and really learn the framework. That's why the identifiers is important and why you may seen different results in different OSes and devices. If you set the identifier then you can guarantee you can find it. It also helps to have multiple levels rather than starting each search from the app. You can also print out the debug of each element you find as opposed to the entire app. print("(controlsView.debugDescription)"). I hope it helps.
– David J
Nov 10 at 12:16
Yes, unfortunately, the view hierarchy may change based upon certain conditions, which is I had to give up screen recording and really learn the framework. That's why the identifiers is important and why you may seen different results in different OSes and devices. If you set the identifier then you can guarantee you can find it. It also helps to have multiple levels rather than starting each search from the app. You can also print out the debug of each element you find as opposed to the entire app. print("(controlsView.debugDescription)"). I hope it helps.
– David J
Nov 10 at 12:16
add a comment |
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53231748%2fxcode-uitesting-test-for-enabled-segmented-control-fails-in-ios12%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password